3.0 Release Notes
Dec 31, 2025
Lakehouse Optimizer 3.0 is a major release that delivers a substantially expanded Optimization Review experience, new incident and recommendation types, a redesigned Usage Distribution visualization with percentile-based metrics, and broad improvements to Forecasting, Recommendations, and Workspace Cost pages. This update also introduces Unity Catalog agent storage for DBFS-disabled environments, enhanced consumption billing support for DBCU plans and Reserved Instances, and significant performance optimizations across the platform.
Optimization Insights
Launched Optimization Review (beta) with comprehensive before/after comparison for Jobs, Pipelines, SQL Warehouses, and Notebooks, including new APIs for each workload type
Added Data Usage statistics (per-run and daily/annualized) to the Jobs summary and details panels, with a new Data Unit Cost KPI panel
Introduced median vs. average toggle for cost and duration statistics across all Optimization Review views
Added pattern-matching job search across all workspaces and refactored filters for improved UX with persistent selections
Redesigned Daily/Annualized view with tabs, moved "What's Included" to the top, and added clickable navigation links from summary to detail views
Fixed permission issue preventing non-admin users from searching by job name, and resolved multiple workspace service request duplication
Workflows & Operations
Added summary KPIs for all run types and scaling of data units (KB/MB/GB/TB) on job runs pages
Enabled prefix search for job run IDs and job IDs in workflow runs filter
Added cluster instance ID linkage for the autoscaling timeline
Improved performance for workflow data pipeline processing and pipeline workflow run cost analysis
Spend Insights & Forecasting
Saved number of tasks in workflow run cost analysis table and refactored daily data availability logic with migration to the new table
Added cutoff date support for Forecasting Cost & Data Usage charts and display of real Databricks cost vs. VM estimated cost in job runs
Added forecaster to the same Docker network as LHO for deployment integration
Fixed Forecast month exceeding chart boundaries and monthly commit line overflow
Fixed Forecasting tags-report endpoint performance taking 20+ seconds per tag
Optimization Guidance
Added configurable date picker on the second page, category in URL for deep linking, and banner on the main screen
Added "created at" and "last update" timestamp columns for incident policies in the Recommendations view
Updated "View Cluster Instance Activity Histograms" link to "View Cluster Instance Usage Distribution" and repositioned "Open in Trends"
Added application ID display as a column in cluster instance from runs
Fixed suboptimal spend showing old values when changing period and columns filter minimum selection count
Updated Recommendations info text on Runs page, fixed N/A impacted Tag Groups display, and corrected "Open in Databricks" option availability
Governance & Alerts
Added SQL Warehouse Auto Shutdown Timeout incidents and interface support for Auto Stop missing and above threshold
Introduced Databricks Platform SKU cost incidents for monitoring platform-level cost anomalies
Added new incident and recommendation type for jobs with high cluster setup overhead
Added support for custom percentiles when generating over/under/imbalanced-provisioning incidents with updated root cause formatting
Added tracking for incident policy creation, updates, and deletions
Updated Incident Policies page styling with new tab design, improved menu color highlighting, and fixed incident email opt-out link
Fixed user role permissions preventing non-admins from configuring or deleting incident policies
Fixed SQL Warehouse incident action not opening the workload, wrong date selection navigating from incidents, unit dropdown showing when only one unit is supported, and filter reset on close
Resource Usage Visibility
Introduced percentile-based CPU/Memory usage distribution visualization with single executor, cluster-wide, and merged views
Added configurable max/median/average displayed values for the Cluster Instance page and CPU/Memory utilization across all executors
Added link to cluster instance from autoscaling timeline
Migrated APC clusters tab to UTC date handling
Added automatic cluster ownership update when the original creator no longer exists in the workspace
Fixed Activity Histograms tooltip missing in merged chart for over 20 executors and displaying values from the previous point
Fixed Usage Distribution single executor view defaulting to wrong executor, cluster view tooltip mismatch, and start/end time display to include the day
Fixed cluster instance response for runs removed from Databricks and excessive shutdown timeout recommendations
Cost Visibility
Added cross-page navigation between Cost Over Time and Workspace Cost by selected resource and provider tab
Added sorting direction indicators on Workspace Cost resource table with default DESC sorting from Cost Over Time navigation
Fixed missing graphic when navigating to Cost Over Time from Workspace Cost and CSV export including wrong workspace data
Fixed CSV export time interval mismatch with selected date range and VM Disks naming in export
Serverless
Implemented serverless Jobs trends view with cost and run analytics
Enhanced serverless jobs query history data loading performance
Billing & Consumption
Added Azure Reserved Instance (RI) plan discount and cost resolution support
Added Databricks unit plan usage tracking during Azure consumption loading and inferred discount handling for legacy pricing
Added new cost categories: AI Functions, Agent Bricks, Base Environments, Data Quality Monitoring, and Delta Sharing
Added ability to retrieve all Databricks charges from billing usage and display Modern vs. Legacy on the consumption page
Deployment & Setup
Added support for storing LHO Agent in Unity Catalog when DBFS is disabled, with removal of Workspace Files init script support
Added support to store and display DBFS status of Databricks workspaces
Added volume init script overrides for environments requiring permission grants
Pushed deployment scripts archive to blob storage on release
Fixed global init script not being disabled on init script failure
Fixed search in Workload Provisioning for All-Purpose Compute and page size reset issues
Fixed agent queue client being called synchronously during startup, causing cluster start failures
Updated Workspace Provisioning default page size to 100 and improved tooltip descriptions for deployment alerts
Fixed provisioning loading state showing previous workspace while loading and non-admin access to provisioning pages
Navigation & Usability
Added timezone (UTC) badge to all date pickers across the application
Added NProgress loading bar for page transitions
Implemented daily LHO User Activity Report scheduling
Removed deprecated "Cluster Instances Activity Histogram" menu link
Fixed Settings navigation menu highlighting wrong item and datepicker highlighting wrong option
Fixed empty tooltip on Trend view and embedded discount plan information in tooltips
Reporting & Visual Clarity
Updated chart X axis to show units for 1st, 10th, and 20th of each month and rotated Y axis labels for consistent direction
Abbreviated numbers over 1000 on Pools main pages (e.g., $1,345 displayed as $1.35k)
Added duration and count values displayed with comma separators for day units
Fixed chart data points not ordered by time and display of data usage in dynamic sizing
Fixed cluster instance graph "load more" behavior and cluster trends filter button state after refresh