If a signed-in Lakehouse Monitor (LHM) user does not have access to any Azure Subscription and/or Databricks Workspace, but the LHM admin still wants to provide read rights to that particular user (or group), the admin can use publicSubscriptionMetadata LHM config file to make subscriptions and workspaces accessible in LHM. This is

As an application-wide setting. Use , use publicSubscriptionMetadata.csv file to provide read-only access to Subscriptions and Workspaces to any user that is able to login to LHM App.

...

Public

...

Lakehouse Monitor (LHM) keeps public and private metadata files which contain Subscriptions and Databricks workspaces information (i.e. IDs).

Private file is used only by consumption library to load consumption data only for these workspaces. Workspaces are only added to the file by using user privileges, service principal privileges and workspaces from the public file (see more details below).

• subscriptionMetadata.csv

...

Subscriptions and Workspaces

LHM configuration file publicSubscriptionMetadata.csvis used in order to expose subscriptions and workspaces to all users who use LHM app regardless of the access rights of that user that are set in Azure or Databricks.

This file must be manually generated in storage account under a folder directory with the name as the subscription id, in a file named publicSubscriptionMetadata.csv. Public file is only read by BPLM LHM app.

This file is used to allow signed-in users to list workspaces and see reports in LHM even if that particular user does not have a user rights configured the tenant that manages the subscription for Databrickson that tenant.

For example, the user signs in to LHM with an external AD account registered with a tenant from another Microsoft AD subscription, thus when a the signed-in user doesn’t have the sufficient rights to read workspaces from the subscription that also manages the Databricks workspaces. In order to allow this user to read workspaces that the admin deemed “public”, LHM uses the publicSubscriptionMetadata.csv to expose these workspaces to this external user.

Manual Public Subscriptions and Workspaces Configuration

Step 1 – Define Public Subscriptions manually in cloud storage

• use cloud storage path from LHM Settings page

  • 

• define in blob storage a file named

...

displayName,subscriptionId,tenantId
Blueprint Data Engine,a63c1e51-40ae-4a34-b230-bf80e132c05c,12e2dd65-5024-44c2-83b5-3ca21c04ef0e

...

Step 2 – Create a folder with subscription-id as name in cloud storage

In this folder, we have two files

• (1)publicSubscriptionMetadata.csv

  • create the file manually

  • add metadata for workspaces that you want to consider “public”

  • example:

  • Code Block

    displayName,isPremium,workspaceHost,workspaceId,workspaceResourceGroupId,workspaceResourceId ca-adb-test-workspace,false,adb-511420607229897.17.azuredatabricks.net,511420607229897,/subscriptions/a63c1e51-40ae-4a34-b230-bf80e132c05c/resourceGroups/databricks-rg-ca-adb-test-workspace-loj4oc72jjfum,/subscriptions/a63c1e51-40ae-4a34-b230-bf80e132c05c/resourceGroups/cost-analyzer-resources/providers/Microsoft.Databricks/workspaces/ca-adb-test-workspace

• (2) subscriptionMetadata.csv

  • created automatically by LHM App

  • generated when we create or update a schedule or when we run on demand consumption loading.

    • see LHM Consumption Management page

...

Metadata Files containing

subscriptionMetadata.csv

• specifies for which workspaces from a given subscription we load consumption data

• update the private file every time user click on run now or save/update scheduler , in my opinion our consumption loader should only read private file. That file should always contain up to date list with workspaces.

• file is updated when consumption loading is run

  • file updated when we create or update a schedule or when we run on demand consumption loading.

• the file will be updated with the following workspaces list:

  • (1) workspaces seen by using rights provided by managed identity or service account

  • (2) workspaces seen by using rights provided by signed-in user

  • (3) workspaces already configured in publicSubscriptionMetadata.csv

• concatenated list of workspaces from (1) ++ (2) ++ (3) is saved in subscriptionMetadata.csv

• config: AUTO_UPDATE_PRIVATE_METADATA_FILE

  • use flag if we should update metadata file or not

  • AUTO_UPDATE_PRIVATE_METADATA_FILE -> default True.

  • if true – file is updated automatically

  • if false – file is updated manually

  • file is always used by Consumption Management module regardless if this config is true or false