3.4 Release Notes
Lakehouse Optimizer 3.4.0 is about closing the loop from seeing spend to acting on it: spot a change, trace it to the workload responsible, and fix the waste — without leaving the app.
Release highlights
Cost Over Time by Tag — chargeback and tag-coverage reporting over any date range, with untagged spend traced to its source.
SQL Warehouse monitoring & auto-stop — instance-level visibility plus managed auto-stop that stops idle warehouses even when open BI connections defeat the native setting.
Genie cost visibility — Genie spend surfaced as its own category, from the Cost Over Time timeline down to each workspace.
SQL Warehouse trends — trendlines for uptime, idle/active ratio, and query volume per warehouse.
Also in this release: more complete and faster workflow run costs, external data source visibility on workflow tasks, cost alert thresholds per policy, richer forecasting plan details, hardened workspace monitoring, and a broad usability pass.
Cost Over Time by Tag
The challenge — Most organizations are mid-journey on tagging. Teams adopt tags at different times, chargeback reports built on fixed windows hide recent progress, and untagged spend is a black box nobody owns.
What 3.4 delivers — Tagging becomes a reporting tool: track cost by tag over any scope and date range, measure tag coverage toward 100%, and trace untagged spend to its source — starting from the exact day a team begins tagging.
Why LHO — No fixed reporting windows, and Untagged is not a dead end: it separates genuinely untaggable platform spend from workloads teams simply haven't tagged yet.
Tag breakdown tab — a new Cost Over Time tab charting spend by tag value over time.
Untagged drill-down — untagged spend expands into a per-workspace breakdown, separating untaggable platform services from workloads that still need tags.
Combine case variants — when several teams adopt tagging independently, the same tag arrives in many spellings:
Projectandproject, values likeEBT,ebt,Ebt. Billing data treats each variant as a different tag, so one project's cost fragments into lines nobody can total — and teams report by tag, not by workspace. The new toggle rolls case variants together, so a project reads as a single line, with no re-tagging campaign required.Readable at any scale — busy tags show their top values with the rest rolled into "Other", so charts stay legible and no spend disappears.
Chart drill-down and CSV export — click a bar to focus on a tag, export the table for reporting.
Forecasting hand-off — filters carry over when arriving from Forecasting, so a spike investigation doesn't restart from scratch.
Totals you can defend — month totals on the Tag view match the other cost views.
Genie & AI Cost Classification
The problem — Genie and other newer AI products were lumped into a generic "AI Other" category, hiding what each one actually costs.
What 3.4 delivers — Genie spend as its own line, from the cost timeline down to each workspace, with every AI product reporting its own category.
Genie gets its own category — Genie usage appears as its own line instead of "AI Other".
Model-serving costs — model-serving cloud spend no longer lands in "AI Other".
Fixed workspace cost totals so every AI category is included.
Note: warehouse compute that runs Genie's queries continues to appear as SQL warehouse cost.
SQL Warehouses
The challenge — Idle warehouses are one of the most common sources of Databricks waste, and the usual fix — the platform's native auto-stop — silently fails when BI tools hold connections open. The warehouse never looks idle to the platform, so it never stops, and no team can control every dashboard that connects.
What 3.4 delivers — Full instance-level visibility (cost, uptime, queries) plus managed auto-stop: LHO watches actual query activity and stops warehouses that are idling, regardless of open connections.
Why LHO — A remediation the platform alone cannot make — including stops below the native minimum — closing the detect → recommend → act loop in one product.
⭐ Spotlight: Auto-stop management
Warehouses that serve dashboards rarely stop on their own. BI tools keep connections open by design — connection pooling avoids costly reconnects — so the platform sees an "active" warehouse even when no query has run for hours, and native auto-stop never fires. The teams paying for it can't fix this at the source, because they don't control every dashboard that connects.
LHO takes over the decision. It reads the warehouse's actual query history and stops any warehouse that has run nothing for longer than your idle threshold — regardless of open connections, and even below the platform's own minimum timeout.
Two settings, and it's running: the idle threshold (how long without queries before stopping) and the check frequency (how often LHO looks).
In practice, teams enable it fleet-wide with a conservative threshold and then forget it's there: dashboards keep working, warehouses stop when truly idle, and the savings accrue quietly month after month. It ships off by default — nothing ever stops without your explicit opt-in.
Instance-level reporting — every warehouse instance listed with cost, uptime, query count, and its auto-stop timeout.
Warehouse trends — trendlines for uptime, idle/active ratio, and query volume per warehouse.
Safe by design — auto-stop can go below the platform's five-minute minimum, and never interrupts a running query.
High idle time detection — incidents and recommendations flag warehouses idling past a threshold you set, per workspace and warehouse type.
Search and navigation — find any instance by its ID, jump to its workloads, and see recommendations per instance.
Fixed idle-time measurement discrepancies, instance counts, and proportional instance cost.
Data Sources & Vendor Consolidation
The challenge — Workflows pull data from many external systems, and what that data movement costs is usually invisible — which makes consolidation decisions guesswork.
What 3.4 delivers — Every workflow task now shows the external data sources feeding it, with per-source volumes and a summary view: the groundwork for spotting redundant platforms worth retiring.
Data sources on workflow tasks — per-source columns and a Data Sources Summary view.
Clearer naming — "Dependent Vendor" is now called "Data Sources" across run views.
Honest attribution — values that cannot be attributed to a single source are explicitly marked as shared.
Serverless coverage restored — data usage for serverless jobs is back, and pipelines break it down per update.
Fixed vendor names being cut off, overlapping badges, zero-cost sources, and defaults for serverless-only environments.
Spend Insights & Forecasting
The challenge — Budget owners juggle commitments, budgets, and actuals across views that didn't always show plan details — or agree with each other.
What 3.4 delivers — A clearer commitments-versus-actuals picture: expanded plan cards, exact-date forecasting, and numbers that match between chart, KPIs, and tooltips.
Expanded cost-plan cards — Databricks and Cloud plan details are visible right where spending decisions are made.
Custom date interval — pick the exact dates the forecast should cover.
Numbers that agree — KPI totals match the chart, and tooltips match the displayed amounts.
Budget context — the Total Cost tooltip shows the budget and cost breakdown behind the number.
Fixed daily-view layout, second-half-of-year forecasts, badge overlaps, and light-theme visibility.
Cost Over Time & Workspace Cost
The challenge — Cost investigations stall when you can't change time granularity or drill from a number to the workload behind it.
What 3.4 delivers — A smoother investigation path: a full year of daily costs, the cloud-versus-Databricks split, faster charts, and direct links from cost cells into the workloads behind them.
Why LHO — Spot the spike, switch to weekly, click through to the job that caused it — minutes, not meetings.
365-day daily view — a full year of daily costs rendered as a line chart.
Cloud vs Databricks split — the percentage split is visible on cost pages.
Straight to Workloads — workspace cost cells link directly into the workloads behind them.
Fixed double-counted feature costs, tooltip totals, month-over-month display, and column sorting.
Incidents & Policies
The problem — Cost problems used to surface on the month-end bill, and a single global alert threshold cannot fit both a sandbox and a production estate.
What 3.4 delivers — Early warning you can calibrate: per-policy thresholds, severity tags, exportable incident lists, and proactive system health alerts.
Per-policy cost thresholds — a $500-a-day sandbox and a $50,000-a-day production estate can't share one definition of "unusual spend": tuned for one, alerts either spam the other or stay silent through a real spike. Each policy now carries its own thresholds, so every environment sets its own sensitivity — strict where the money is, relaxed where experimentation is expected — and alerts stay meaningful enough to act on.
Severity tags on incidents and policies make triage faster.
Export to CSV for cost and performance incidents.
learn more here: Export to CSV: taking your LHO data into the tools you already use
Low-disk alert — a weekly system check warns before the host runs out of disk space.
Date-range links on idle-time incidents jump straight to the period in question.
Fixed stale monthly incidents, filter resets, clipped labels, and page scrolling.
Deployment & Administration
The problem — Operating the platform shouldn't require container expertise, or manual digging to answer "is everything healthy?"
What 3.4 delivers — Lighter deployment options, self-service health visibility, and safer data lifecycle controls.
Dockerless deployment — install and update without a container runtime, including a standalone update script.
System health at a glance — host status and an on-demand database status check on the System Updates page.
Safer data purging — long-horizon data cleanup gained guardrails against accidental deletion.
Faster help — a reorganized support menu with a Get Urgent Help option.
Fixed the license expiry indicator and email noise from test environments.