✨ Release Highlights

• estimated costs support across all app

  • LHO is able to show estimated costs instead of real costs all across the application

  • LHO uses publicly available standard rates for Cloud (Azure or AWS) and Databricks resources in order to accurately estimate cost of usage at the most granular level available

  • LHO estimated costs are available for all clusters types, workflows, delta live tables

  • if discount rates are made available to LHO by the users, then these can also be leveraged by the estimation algorithm

Open

Workloads // Workflows

• 8642 - Expose attempt_number in performance incidents related to job runs

Open

❤️ Incidents // Performance

• 8659 - expose Cluster Idleness Incident

  • This policy specifies an upper limit to the time allowed to a cluster instance to idle before an incident is generated.

📊 Analysis Optimizations

• 8289 - Add a resource management and orchestration service for analysis and cluster events.
  These changes are necessary to segregate the analysis of cluster resources (such as cluster events, timeline, and estimated costs) from the telemetry-driven analysis (including jobs, pipelines, notebooks, and idleness).

🪩 UX Optimizations

• 8681 - widen horizontal scroll bar to enhance horizontal scrolling

• 8407 - add badge for email configuration valid / invalid and enhanced documentation

• 8568 - regroup management pages under one tab and add pages for Consumption, Telemetry, Detection, Estimated Costs

• 8788 - Adapt labels and fix sorting

🎛️ Configuration & Provisioning

• 8727 - Provisioning / Workflows add cancel search feature

Operational Management // Telemetry Analysis

• 8633 - Add repair for clusters missing TERMINATING event

• 8633 - Mark cluster as terminated when adding the artificial TERMINATING event

• 8759 - Recompute cluster timeline for clusters with incomplete segments
  Clusters that have the following events in order ADD_NODES_FAILED, RUNNING, UPSIZE_COMPLETED, have the first timeline segment missing. The timeline has to be recomputed for these clusters.

📈 Operational Management // Consumption (Cost) Analysis

• 8710 - consumption loading / history run not showing durations intuitively
  Added a field called "level" to each run, so we can show nested runs in UI

Miscellaneous

• 8726 - Log duration for list clusters and list jobs in provisioning page.

🐞 Fixed Bugs

• 8721 - Inconsistent job run analysis state (analysisType=FINAL but status=PENDING)

• 8443 - Performance and Orchestration are missing from Spend Breakdown chart

• 8168 - Telemetry analysis history shows regular job runs in the Pipeline-Maintenance-Job and One-Time-runs tab

• 8750 - Fix job analysis processing for attempted runs

• 8749 - Js error raised when changing cluster selection on Pool Activity

Documentation

• Configuring Email Notifications