Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Needed files

View file
namepython_provisioning.py

...

The following provides a comprehensive guide to understanding and utilizing the capabilities of the Azure Function for LHO Permissions setup script within your Azure environment.

Info

This page contains:

  • script to grant permissions to Lakehouse Optimizer service principal

  • instructions to setup an automated mechanism to grant same permissions to LHO for newly added assets in a Databricks Workspace

The service principal used by LHO requires permissions to be granted to it in order to access both Databricks Workspaces and assets managed by workspaces. These access rights can be granted via the python_provisiong.py script, which requires an admin Databricks account to grant permissions.

If new assets (workflows, notebooks, clusters) are added in a workspace, then the LHO service principal needs permission to access those assets as well. This can be done automatically by adding the previous script as an Azure Function that runs recurrently on scheduled basis.

Prerequisites

Setup azcli

Install azcli core tool from Azure on your local machine

If core tools have been installed correctly, then func --version should work and return the current version.

Login with azcli

Run az login

How to create the Azure Function

Step 1. In the Azure Portal create a new resource group
This resource group is used to group all assets related to this Azure Function.

Step 2. Create a new function locally

Run the following in command line.

Code Block
func init az_functions --python -m V1
cd az_functions

Step 3. Edit the requirements.txt file generated and append the following to it

Code Block
certifi==2023.5.7
charset-normalizer==3.1.0
databricks-sdk==0.1.8
idna==3.4
requests==2.28.2
urllib3==1.26.16

Step 4. Create the storage account needed for the function

Code Block
az storage account create --name lhmgrant --resource-group <YOUR_RESOURCE_GROUP> -sku Standard_LRS

Step 5. Create a new function based on the Timer trigger template

Code Block
func new --name lhm-grant --template "Timer trigger"

Step 6. Download the python_provisioning.py and azure_main.py into the new function directory lhm-grant

View file
nameazure_main.py
View file
namepython_provisioning.py

Step 7. Edit azure_main.py and provide needed information for the grant code

Step 8. Change the cron schedule inside lhm-grant/function.json

func init will create a file function.json

Step 9. Change the lhm-grant/__init__.py file to call the grant function

Code Block
import datetime
import logging

import azure.functions as func
from .azure_main import main as m


def main(mytimer: func.TimerRequest) -> None:
    m()

Step 10. Create the Azure Portal FunctionApp

Code Block
az functionapp create --resource-group <YOUR_RESOURCE_GROUP> --runtime python --runtime-version 3.7 --functions-version 4 --name lhm-grant --os-type linux --storage-account lhmgrant --consumption-plan-location <CONSUMPTION_PLAN_LOCATION>

Step 11. Publish the app

Code Block
func azure functionapp publish lhm-grant