Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The Lakehouse Optimizer empowers users to monitor and improve their Lakehouse infrastructure by configuring incidents of interest that identify inefficiencies in cost, performance, and operational metrics. Incidents are displayed in the Incidents section of the app, providing actionable insights to optimize resource utilization and expenditures.

...

...

Incident Configuration Steps

  1. Under the Settings menu, select Settings / Incident Policies

    Incident Policies.png
    1. Incidents can be defined for within Subscriptions, Workspaces, Workflows, All Purpose Compute, Delta Live Tables, SQL Warehouses, Pools, and Job Compute areas.

    2. Each incident created has its own incident policy

  2. Select the area of interest to create an incident under (ex. Workflow).

    Workflow.pngImage Added


  3. Select category - Cost Control or Performance

    image-20250116-234155.pngImage Added

  4. Select sub-category from dropdown menu (ex. Over Provisioning)

    Category and Sub Category.png


  5. Select sub-option within Sub-Category (ex. Cluster CPU Over Provisioning)

    Sub Category Details.pngImage Added

  1. Select +ADD button under Incident Rules section

    image-20250116-235453.pngImage Added

  1. Specify threshold for the incident rule (ex. Cluster CPU under 60% for any run) and save changes.

    Screenshot 2025-01-16 155708.pngImage Added

    a. multiple rules can be set for each incident policy (ex. if a user wants an incident created each time Cluster CPU for any run is under 60%, 65%, or 70%)

    Screenshot 2025-01-16 160419.pngImage Added

b. When an incident rule has been met, an Incident Ticket is created automatically (and corresponding incident will appear in the Incidents view.

...

The following incidents are configurable in LHO:

...