AWS CloudWatch canary for LHO Permissions setup script

Owned by Denis Pitul, created
Last updated: Aug 10, 2023 by Ciprian LUCACI
3 min read

This page contains:

  • script to grant permissions to LHO service principal

  • instructions to setup an automated mechanism to grant same permissions to LHO for newly added assets in a Databricks Workspace

 

The service principal used by LHO requires permissions to be granted to it in order to access Databricks Workspaces and those assets managed by workspaces. These access rights can be granted via the python_provisiong.py script which requires an admin Databricks account in order to grant permisions.

 

If new assets (workflows, notebooks, clusters) are added in a workspace, then the LHO service principal need to be granted permission to those assets as well. This can be done automatically by adding the previous as script as an AWS Canary that runs recurrently every 15 min.

How to create the AWS Canary

Step 1. Create a python virtualenv
On your local machine create a virtual env in order to prepare the canary archive.

mkdir lhm-grant cd lhm-grant python3 -m venv .venv source .venv/bin/activate

 

Step 2. Download the requirements.txt into the created lhm-grant folder

 

Step 3. Install required packages

pip install -r requirements.txt

 

Step 4. Copy all the installed libraries into a folder called python

mkdir python cp -rfv .venv/lib/python3.7/site-packages/* python/

 

Step 5. Download aws_main.py and python_provisioning.py into the python directory

 

 

 

Step 6. Modify aws_main.py script and provide the needed information for the grant code in the section marked for change

 

Step 7. Create an archive with the python folder

 

Step 8. Upload the archive to s3

 

Step 9. Create CloudWatch canary from s3 bucket artifact

  • name you canary as lhm-grant (or any other name that you want to use)

  • select the S3 location for the archive

  • set the lambda handler (your entry point for the script) to aws_main.handler

  • set the schedule to continuously with intervals of 15 mins

    • recommended configuration

 

Step 10. Check logs of the python_provisioning.py script run

Open the list of Synthetic Canaries and select the canary configured previously (e.g. lhm-grant) in order to select a particular run and view the logs.

To check the logs of a run you will download the artifacts archive from that run, unzip it and there should be a file called python_provisioning.log