/
Azure Function for Lakehouse Optimizer Permissions setup script

Azure Function for Lakehouse Optimizer Permissions setup script

Owned by Denis Pitul
Last updated: Apr 30, 2024 by Megan Novak
2 min read

The following provides a comprehensive guide to understanding and utilizing the capabilities of the Azure Function for LHO Permissions setup script within your Azure environment.

 

This page contains:

  • script to grant permissions to Lakehouse Optimizer service principal

  • instructions to setup an automated mechanism to grant same permissions to LHO for newly added assets in a Databricks Workspace

 

The service principal used by LHO requires permissions to be granted to it in order to access both Databricks Workspaces and assets managed by workspaces. These access rights can be granted via the python_provisiong.py script, which requires an admin Databricks account to grant permissions.

If new assets (workflows, notebooks, clusters) are added in a workspace, then the LHO service principal needs permission to access those assets as well. This can be done automatically by adding the previous script as an Azure Function that runs recurrently on scheduled basis.

Prerequisites

Setup azcli

Install azcli core tool from Azure on your local machine

If core tools have been installed correctly, then func --version should work and return the current version.

Login with azcli

Run az login

How to create the Azure Function

Step 1. In the Azure Portal create a new resource group
This resource group is used to group all assets related to this Azure Function.

 

Step 2. Create a new function locally

Run the following in command line.

func init az_functions --python -m V1 cd az_functions

 

Step 3. Edit the requirements.txt file generated and append the following to it

certifi==2023.5.7 charset-normalizer==3.1.0 databricks-sdk==0.1.8 idna==3.4 requests==2.28.2 urllib3==1.26.16

 

Step 4. Create the storage account needed for the function

az storage account create --name lhmgrant --resource-group <YOUR_RESOURCE_GROUP> -sku Standard_LRS

 

Step 5. Create a new function based on the Timer trigger template

 

Step 6. Download the python_provisioning.py and azure_main.py into the new function directory lhm-grant

 

 

Step 7. Edit azure_main.py and provide needed information for the grant code

 

Step 8. Change the cron schedule inside lhm-grant/function.json

func init will create a file function.json

 

Step 9. Change the lhm-grant/__init__.py file to call the grant function

 

Step 10. Create the Azure Portal FunctionApp

 

Step 11. Publish the app