The service principal used by LHM requires permissions to be granted to it in order to access Databricks Workspaces and those assets managed by workspaces. These access rights can be granted via the python_provisiong.py script which requires an admin Databricks account in order to grant permisions.

If new assets (workflows, notebooks, clusters) are added in a workspace, then the LHM service principal need to be granted permission to those assets as well. This can be done automatically by adding the previous as script as an Azure Function that runs recurrently on scheduled basis.

Needed files

Prerequisites

Setup azcli

Install azcli core tool from Azure on your local machine

• https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?tabs=macos%2Cportal%2Cv2%2Cbash&pivots=programming-language-python

• https://learn.microsoft.com/en-us/cli/azure/install-azure-cli

If core tools have been installed correctly, then func --version should work and return the current version.

Login with azcli

• https://learn.microsoft.com/en-us/cli/azure/get-started-with-azure-cli#how-to-sign-into-the-azure-cli

Run az login

How to create the Azure Function

Step 1. In the Azure Portal create a new resource group
This resource group is used to group all assets related to this Azure Function.

Step 2. Create a new function locally

Run the following in command line.

func init az_functions --python -m V1
cd az_functions

Step 3. Edit the requirements.txt file generated and append the following to it

certifi==2023.5.7
charset-normalizer==3.1.0
databricks-sdk==0.1.8
idna==3.4
requests==2.28.2
urllib3==1.26.16

Step 4. Create the storage account needed for the function

az storage account create --name lhmgrant --resource-group <YOUR_RESOURCE_GROUP> -sku Standard_LRS

Step 5. Create a new function based on the Timer trigger template

func new --name lhm-grant --template "Timer trigger"

Step 6. Download the python_provisioning.py and azure_main.py into the new function directory lhm-grant

Step 7. Edit azure_main.py and provide needed information for the grant code

Step 8. Change the cron schedule inside lhm-grant/function.json

func init will create a file function.json

Step 9. Change the lhm-grant/__init__.py file to call the grant function

import datetime
import logging

import azure.functions as func
from .azure_main import main as m


def main(mytimer: func.TimerRequest) -> None:
    m()

Step 10. Create the Azure Portal FunctionApp

az functionapp create --resource-group <YOUR_RESOURCE_GROUP> --runtime python --runtime-version 3.7 --functions-version 4 --name lhm-grant --os-type linux --storage-account lhmgrant --consumption-plan-location <CONSUMPTION_PLAN_LOCATION>

Step 11. Publish the app

func azure functionapp publish lhm-grant