The Lakehouse Optimizer empowers users to monitor and improve their Lakehouse infrastructure by configuring incidents of interest that identify inefficiencies in cost, performance, and operational metrics. Incidents are displayed in the Incidents section of the app, providing actionable insights to optimize resource utilization and expenditures.

Incident Configuration Steps

1. Under the Settings menu, select Settings / Incident Policies

   

   1. Incidents can be defined within Subscriptions, Workspaces, Workflows, All Purpose Compute, Delta Live Tables, SQL Warehouses, Pools, and Job Compute areas.

   2. Each incident created has its own incident policy
      

2. Select the area of interest to create an incident under (ex. Workflow).

   

   
   

3. Select category - Cost Control or Performance

   

4. Select sub-category from dropdown menu (ex. Over Provisioning)

   

   
   

5. Select sub-option within Sub-Category (ex. Cluster CPU Over Provisioning)

   

6. Select +ADD button under Incident Rules section

   

7. Specify threshold for the incident rule (ex. Cluster CPU under 60% for any run) and save changes.

   

   a. multiple rules can be set for each incident policy (ex. if a user wants an incident created each time Cluster CPU for any run is under 60%, 65%, or 70%)

   

b. When an incident rule has been met, an Incident Ticket is created automatically (and corresponding incident will appear in the Incidents view.

8. Every incident rule for a given Incident Policy automatically has email notifications turned on. However, by default no email group is tied to the incident rule, thus email notifications will not be sent until an email group(s) is selected for a given incident rule.

   

   a.

The following incidents are configurable in LHO:

Related articles