The Lakehouse Optimizer empowers users to monitor and improve their Lakehouse infrastructure by configuring incidents of interest that identify inefficiencies in cost, performance, and operational metrics. Incidents are displayed in the Incidents section of the app, providing actionable insights to optimize resource utilization and expenditures.

Open

Incident Configuration Steps

1. Under the Settings menu, select Settings / Incident Policies

   Open

   

   1. Incidents can be defined within Subscriptions, Workspaces, Workflows, All Purpose Compute, Delta Live Tables, SQL Warehouses, Pools, and Job Compute areas.

   2. Each incident created has its own incident policy
      
      

2. Select the area of interest to create an incident under (ex. Workflow).

   
   
   

3. Select category - Cost Control or Performance

    

4. Select sub-category from dropdown menu (ex. Over Provisioning)

   
   
   

5. Select sub-option within Sub-Category (ex. Cluster CPU Over Provisioning)

6. Select +ADD button under Incident Rules section

7. Specify threshold for the incident rule (ex. Cluster CPU under 60% for any run) and save changes.

   a. multiple rules can be set for each incident policy (ex. if a user wants an incident created each time Cluster CPU for any run is under 60%, 65%, or 70%)

8. Every incident rule for a given Incident Policy automatically has email notifications turned on. However, by default no email group is tied to the incident rule, thus email notifications will not be sent until an email group(s) is selected for a given incident rule.

9. To assign an email group(s) to an incident rule, select the Email Group dropdown and select email group(s) (ex. Leadership Team, lho-support)

10. To configure additional incident policies, you can navigate to Incident Policies under Settings or select the dropdown at the top of the screen and choose a different area of interest.

     

The following incidents are configurable in LHO:

Email Group Configuration Steps

1. To create a new email group, select Settings / Email Notifications

2. Select +Add Group for Custom Groups

3. Specify group name (ex. Blueprint Test) and Save

4. Select +Add Email for the email group

5. Specify an email account and Add

   1. Repeat process to include additional email accounts for a group

6. See steps #8 and #9 under Incident Configuration Steps above on how to assign an email group to an incident policy.

Related articles