3.5 Release Notes
Version: 3.5.0
Period: July 14, 2026 — August 4, 2026
Lakehouse Optimizer 3.5.0 puts Executive Insights at the front door of the product: an executive home page that reads your Databricks estate, ranks where money and time can be recovered, and links every opportunity to the action that fixes it. Around the headline: deeper Genie cost visibility, the first All-Purpose Compute views in Optimization Review, a new cluster-contention detection, and passwordless managed-identity deployment on Azure.
Release highlights
Executive Insights — the new landing page for every user: one screen answering What do I have? How am I doing? What should I do about it? — see the Feature Guide below.
Genie cost visibility, deepened — Genie spend broken into subcategories, reachable straight from the cost timeline, with an agents view that reflects reality.
Also in this release: All-Purpose Compute views in Optimization Review, cluster-contention detection with a disk-spillage root cause, a SQL Warehouses refinement pass, richer cost-plan details, Databricks Runtime 18/19 support, and managed-identity authentication end to end.
Executive Insights — Feature Guide
The problem — Executives ask three questions about their Databricks estate: What do I have? How am I doing? What should I do about it? Traditional dashboards answer with charts and leave the interpretation — and the follow-up — to the reader.
What 3.5 delivers — Executive Insights becomes the home page of Lakehouse Optimizer: open the app and this is the first thing you see. It compresses the estate into the few numbers a leadership team actually acts on, then connects each one to the concrete recommendations that move it.
Why LHO — Insights, not dashboards: every element on the page exists to produce a decision, not to display raw data. And it frames optimization in both directions — where to cut waste and where to build strength (serverless adoption, Unity Catalog migration, consolidated orchestration) — a Databricks Centre of Excellence view, not just a cost-cutting tool.
The top row situates you — your Databricks cost plan, Total Workspaces (with a "with telemetry" coverage chip, Unity Catalog adoption, and regions), the Health Score, and Average Daily Cost.
A four-tile KPI band tracks the estate month over month — Cost per TB, Data Usage, Throughput, and Monitored Cost, each with its trend; hover any value to see the current and previous month side by side.
The Health Score card carries the two numbers behind it — Potential Cost Savings and Total Cost.
Three tabs — Cost Savings, Performance, Orchestration — each ranking your workspaces by Suboptimal Cost, so the workspaces worth attention rise to the top.
Opportunity cards lead to action — cost opportunities link one click into the exact, pre-filtered recommendations that fix them; Performance and Orchestration opportunities carry workload counts and guidance.
Pick your month — default is the last full month; the current (partial) month is selectable too.
The savings story is deliberately conservative, built as a funnel: Total Cost (your invoice) → Monitored Cost (the share LHO can analyze and recommend on) → Suboptimal Cost (workloads that actually have recommendations) → Potential Cost Savings (an estimated recoverable range). Savings are computed strictly from cost opportunities — where a defensible figure doesn't exist, none is shown.
The full guide — how each number is computed, how to read the header, and how to go from a tile to a fix — lives in the Executive Insights Walkthrough.
Genie & AI Cost Visibility
The problem — Genie spend became visible as its own category in 3.4, but "Genie" is really several products, and questions like which part of Genie costs what still meant digging.
What 3.5 delivers — Genie cost broken down by subcategory, reachable directly from the cost timeline, and a Genie agents view that reflects what is actually deployed.
Genie subcategories in Cost by Feature — select individual Genie subcategories instead of one combined line.
Straight to Genie from Cost Over Time — the Genie entry links through to its cost detail.
A truthful agents list — deleted agents are marked, and each agent links to Open in Databricks.
Optimization Review — All-Purpose Compute
The problem — Interactive clusters are where quiet waste accumulates, and Optimization Review previously had no dedicated view for them.
What 3.5 delivers — The first All-Purpose Compute views: a fleet-level summary and a workloads tab listing individual clusters, with per-instance detail per KPI.
Optimization Review — All-Purpose Compute
All-purpose compute is where waste hides in plain sight: clusters shared by many users, running between notebook sessions, idling after the last command, or doing driver-side work while the cluster nodes sit unused.
Optimization Review gives you a before-and-after view of every all-purpose cluster — compare any two periods and see cost, uptime, data usage and idle ratio side by side, together with the configuration changes made in between.
Whether you applied an LHO recommendation or made your own change, you can verify it actually worked: the idle ratio should drop and cost per unit of work should improve, even when overall usage grows. No more manual comparisons across browser tabs — the trend, and its explanation, are on one screen.
All-Purpose Compute Summary — a fleet-level overview of your interactive compute.
Workloads tab — individual clusters listed with their key metrics.
Per-instance columns — each KPI can be broken down per instance.
Only monitored workspaces counted — unmonitored clusters do not skew summaries and stats.
Recommendations & Incidents
The problem — When parallel tasks fight over the same cluster's disk, jobs slow down and costs climb — and nothing pointed at the cause.
What 3.5 delivers — A new cluster-contention detection that names the root cause, plus a faster and more consistent Recommendations experience.
Cluster-contention incidents and recommendations — contention is detected and explained down to its disk-spillage root cause.
From policy to incidents — incident policies link directly to the incidents they raised.
SQL Warehouses
The problem — 3.4 introduced warehouse monitoring and auto-stop; day-to-day use surfaced places where the numbers and navigation could be sharper.
What 3.5 delivers — A refinement pass across instance reporting, incident navigation, and data completeness.
Optimization Review — SQL Warehouse
SQL warehouses bill for every minute they're up, not every minute they're working — so a warehouse can look busy while spending most of its uptime idle between queries or waiting out an oversized auto-stop timeout.
Optimization Review extends the same period-over-period comparison to SQL warehouses: uptime, cost and idle ratio per warehouse and per day, alongside the configuration changes that explain any shift.
That turns warehouse tuning — resizing, auto-stop adjustments, workload consolidation — from a guess into a measurable loop: make the change, compare the periods, confirm the idle ratio fell and efficiency improved. It's the holistic warehouse view teams have been asking for, without stitching together query history and billing data by hand.
Ratio bars at a glance — Active/Up Time and Active/Idle Ratio render as visual bars in the instances table.
Idle incidents open in context — an idle-time incident lands directly on the affected warehouse, pre-filtered.
Platform Security & Deployment
What 3.5 delivers — Passwordless operation end to end on Azure, native service identities, and support for the newest Databricks runtimes.
Managed identity across the stack — database access, the monitoring agent, and admin roles can all authenticate without stored passwords.
Databricks Runtime 18/19 support — the collector agent was modernized to run on the newest runtimes.
Related articles
Have questions or need help getting started? Please Contact Us for further discussion. We are here to help you make the most out of your Lakehouse Optimizer experience!