2.28 Release Notes

2.28 Release Notes

 

28 July 2025

🚀 Lakehouse Optimizer 2.28 – What's New

Version 2.28 introduces key improvements to cost visibility, incident management, and overall user experience.


🌟 Highlights

  • New features and functionalities

    • Vendor Consolidation feature has been released, with relevant data collection starting in July 2025. To learn more, see Vendor Consolidation page.

    • “Lakebase” cost category has been added under “Databricks Platform“ feature set on “Cost Over Time“ page.

    • Detailed metrics added for “Apps“ under Workloads > Cost Over Time (entity name, cost, and creator).

    • Detailed metrics added for “Vector Search“ and “Model Serving” under Workloads > Serverless (entity name, cost, and creator).

    • Detailed reporting on untagged in “Cost by Tag Group” for Creator tag.

  • 💰 Cost & Forecasting

    • Improved cost accuracy with enhanced metadata and refined ingestion logic.

    • Stabilized cost charts and improved daily forecasting visuals.

    • Enhanced tooltips and cost KPIs for clearer insights.

  • 🚨 Incidents

    • New Job Duration Exceeded incidents for long-running job detection.

    • Scoped incident rules at subscription/workspace levels for better control.

    • Persisted job run recommendations to support historical optimization.

  • 🎨 UI/UX

    • Refined filters, tooltips, and layouts for greater consistency.

    • Enhanced navigation speed and dropdown responsiveness.

    • Improved visibility of workspace features and serverless KPIs.

  • ⚙️ System Enhancements

    • Introduced serverless KPIs (hourly cost rate and data cost rate).

    • Enhanced DBU SKU parsing and duplicate cluster name handling.

    • Strengthened cluster metadata accuracy and provider tagging.


🔧 Fixes & Stability

  • Corrected missing metadata for clusters and SQL warehouses.

  • Improved API stability for search, incidents, and cost reporting.

  • Reduce thread pool size to 1 in consumption services for optimized resource utilization.


Benefit: This release delivers stronger cost governance, enhanced performance tracking, and a streamlined experience for platform and data teams.

 

Forecasting // Cost & Data Usage widget

image-20250728-154012.png
Cost & Data Usage
image-20250728-154047.png
Cost/TB. Data Usage Cost / Data Usage. Represents the average cost per TB for processed data.
Data Usage is also known as Data Processed.
image-20250728-161443.png
TB/day. Data Usage / Days in Month. Represents the average of processed data per day.

 

KPIs as side panel on Forecasting page

image-20250728-163915.png
image-20250728-163932.png

 

 

Display last 5 runs status on workflows - jobs

image-20250728-161727.png

 

Navigation link "View Cost Breakdown" from Forecasting/Daily view to Forecasting/Workspaces Cost

  • enhanced navigation to support to investigate daily cost spikes

image-20250728-161903.png

 

Vendor Consolidation

image-20250728-162027.png

Distribution of cost per vendors

image-20250728-162324.png
image-20250728-162344.png
image-20250728-162405.png

View dependent vendor query references per run

image-20250728-163627.png
image-20250728-163655.png

 

 

 

Incident to compare daily cost of job

Incident to compare daily cost of job with AVG daily cost of Last Week, Last Month.

Generate daily individual job cost incidents, when the cost of a day increases over the threshold defined in the incident rule compared to the average cost over the previous week/30 days.

For example, if today is a workday (Monday to Friday) compare to
(1) average daily cost from Monday to Friday previous calendar week (starting previous Monday) --> avg over week
(2) take previous 30 days starting with yesterday, and take average daily cost of working days in that period --> avg over month

if today is weekend (Saturday to Sunday)
compare to
(1) average daily cost from Monday to Sunday previous complete calendar week --> avg over week
(2) take previous 30 days starting with yesterday, and take average daily cost of all days in that period --> avg over month

Week starts on Monday

Real cost incidents are generated independently of estimated cost incidents

Current limitations:

  • when the cost of a day from the reference interval is reanalyzed/updated, the cost of the "current days" (i.e. days having the updated day in their reference interval) will not be updated

  • if an incident was created and the average cost of the previous days increases on further re-analyses, the incident will not be healed (deleted)

image-20250728-163052.png

 

Redesigned Wokloads main menu

with individual items for

  • Worklflows

  • All-Purpose Compute

  • Pipelines

  • SQL Warehouses

  • Pools

  • Serverless

  • Databricks Platform

Menu has been simplified to have at the top of the pages only one group of tabs where applicable.

Tabs have the full name always fully visible.

image-20250728-163154.png
Workflows
image-20250728-163310.png
All-Purpose Compute
image-20250728-163331.png
Serverless
image-20250728-163342.png
Pools
image-20250728-163434.png
Databricks Platform
image-20250728-163450.png
SQL Warehouses

 

Updated trendline kpis, reordered and included additional info

with enhanced filtering

  • kpi filters with multiple units selection

image-20250728-163809.png

 

Export to CSV in All-Purpose Compute // Cluster instances

image-20250728-164012.png

 

Support search with whitespaces on workloads

  • support search with whitespace in "AllPurpose/Clusters: name"

  • search by name with whitespace supported SQL Warehouse

  • support DLT pipeline search by name with whitespace

  • support whitespace search by name for one-time runs