Versions Compared

Old Version 1

changes.mady.by.user Denis Pitul

Saved on

compared with

New Version Current

changes.mady.by.user Claudiu Barbura

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In order to monitor the Lakehouse Monitor we will be using existing Azure monitoring solutions (Insights and Alerts).

On the VM hosting your LHM you will be preparing a python environment and cron a python script that checks all the Subscriptions and Workspaces that are available to the Monitor in order to find any issues in retrieving information for them.

Script can be downloaded here <TODO create link>

Code Block
wget link to download

Once the script is in place, it’s time to create the cronjob that runs it.

Code Block
crontab -e

we recommend running the script every 30/60 minutes to be on top of the situation.

Code Block
0,30 * * * * python3 bplm-check.py

The script and adjacent resources

Files present in the archive

  • bplm-check.py - main script

  • .env - environment file

  • requirements.txt - text file used to setup the python environment

Setting up the Application Insight

The script in the archive uses REST requests to query your LHM for information about: subscriptions, workspaces, etc.

Everything it does, including any issues it finds (no workspaces, no clusters, issues in connecting to them to list) it logs. The logs it sends to Azure. For this you’ll have to create a Application Insight in the Azure portal

  1. in the portal, go to Monitor

  2. in the left sidebar go to Insights → Applications

  3. Create a Insight for your LHM deployment (in the same resource group as the other resources to ease locating it later)

...

  • Create a Function App by going to the Azure Portal, searching for “Function Apps” and clicking Create. Follow the rest of the wizard and make sure you create the Application Insights for the Function App as well (this will be needed in the alert creation stage later).

  • Create the system assigned managed identity for your Function App by going to your Function app’s Identity section and setting the Status from Off to On. You need this in order for the script to pull the client secret from the KeyVault.

    Image Added

  • Copy the Object ID from the identity page and go to your KeyVaults Access policies. Here you will create a new policy which will grant List and Get rights for Secrets for the Function Apps Identity (search for it by using it’s object ID.

  • Next you will be creating the local function project. This part of the process takes place in your terminal.
    Make sure the pre-requisites are fulfilled. The following command creates the local project:

    Code Block
    func init <PROJECT NAME> --python # give it your function app name
cd <PROJECT NAME>                 # the func init creates this folder

  • After going into the function folder you will pull the necessary files for the monitoring function and unzip it in the current folder:

    Code Block
    wget <URL TBD>
unzip -j bplm-check.zip

  • The core files unarchived above is being defined by the following 4 files:

    Code Block
    bplm-check/bplmcheck.py
bplm-check/.env
bplm-check/__init__.py
requirements.txt

    The other files in the archive are azure provided files which contain metadata pertaining to the monitor. They are:

    Code Block
    bplm-check/function.json
bplm-check/host.json
bplm-check/readme.md

    Note: the function is based on the TimeTrigger template and is configured to run every 30 mins. The cron expression can be found inside this file

    Code Block
    bplm-check/function.json

  • Your current function folder should contain the following:

    Code Block
    .gitignore
.vscode
bplm-check
getting_started.md
host.json
local.settings.json
requirements.txt

Note: the bplm-check directory contains the files outlined above.

  • At this point, you can publish the function to the Function App you’ve created above by issuing:

    Code Block
    func azure functionapp publish <FUNCTION APP NAME> -b remote

  • Checking the Azure Function App at this point should show you the function you’ve just published.

...

  • Test it out by clicking on it, going to Code + Test and hit the Test/Run button. If you are not seeing the output in the console of this page, you can go to the Monitor page to see the logs. Successful runs look like this.

...

Note: These log messages get publish to the Application Insights that’s tide to the Function App, which is where we’re setting up the alert based on the ERROR message count in these logs.

Setting up the alert

With the information successfully being sent to Azure, you can now set up an Alert.

...