Configure Cluster Scope Init Script Storage

cluster-init-script - is a LHO script that copies the LHO Telemetry Agent jar from DBFS to the storage area accessible by clusters, which can be either in /Volumes or in /Workspace (workspace files) depending on Unity Catalog setup.

Unity Catalog Enabled

Shared-Compute resources are available only with Unity Catalog enabled.

At the moment of this documentation creation, Databricks does not offer an API to upload the cluster-init-script in /Volumes, therefore the admin needs to configure the cluster-init-script by following the next steps.

Step 1) Use the following link to download the LHO cluster-init-script

• https://bplmdemoappstg.blob.core.windows.net/libraries/LHO_VERSION/init_script.sh

  • e.g. https://bplmdemoappstg.blob.core.windows.net/libraries/2.7.2/init_script.sh

Step 2) Prepare storage path in Catalog Explorer

• open Catalog Explorer

• select schema: main/default

• create volume: bp-lakehouse-monitor

Step 3) Upload script to

/Volumes/main/default/bp-lakehouse-monitor/script/

The complete path should look like:

/Volumes/main/default/bp-lakehouse-monitor/script/init_script.sh

Step 4) In Databricks

• open Catalog

• open Metastore details

• open Allowed JARs/Init Scripts

• add a new entry for

/Volumes/main/default/bp-lakehouse-monitor/script/init_script.sh

Grant Account Users Read Access to Volume

• Click on the bp-lakehouse-monitor volume in ‘Catalog Explorer’

• Select Permissions

• Click on Grant

• Select READ VOLUME and add ‘Account Users’ principal

• Click on Grant

Unity Catalog Disabled

This configuration step is done automatically by LHO when LHO Telemetry Agent is updated.

LHO copies the cluster-init-script into the storage area accessible by the cluster, i.e. the Databricks workspace files

/Workspace/bp-lakehouse-monitor/script/init_script.sh

Related articles

• Auto-Pilot Monitoring (AWS)