🎉 Highlights of 2.4.1 Release

• support telemetry reanalysis for finished workflows

• additional columns for reporting tables (failed, skipped, endtime)

• SQL warehouse autoscaling timelines

• Performance Evaluation Report

Overview

• 5897 - add totals on top of widgets for bars on overview page

Open

overview

Reporting // Workflows

• 5214 - support reanalyzing the telemetry on a job run that is already finished

Open

reanalyzing the telemetry on a job

• 4869 - Memory Spillage analysis and report

• 5962 - exposed workflow run end-time in reports

• 4191 - identify tasks that are DLT pipelines with icon

• 5132 - add section with Task Run info in the autoscaline timeline view for Job clusters

• 6051: add Results Search box for Delta Live Tables Pipeline Updates view and for DLT Maintenance Job Runs

• 6082 - add start time and run id to one time runs reports, sort by time, search by name and ids

🖥️ Reporting // All purpose clusters

• [5886] - Include telemetry start time in all purpose cluster timeline API response

• [5925] - cluster and driver memory and cpu tooltips

• 5533 - Add number of failed runs to reports

🖥️ Reporting // SQL Warehouse

• 5914 - Sql Warehouses cluster timeline

🖥️ Reporting // Pools

• 6042 - VM pools cost breakdown in Reports

❤️ Health Diagnostics // Cost

• 5957 - Change label for Health Alerts to Health Diagnostics

• 5935 - remove the expanding functionality from the health alerts cards and display always the entire information

• 5805 - add alert type indicator to cost alerts

• 5807 - improve cost increase section for cost alerts

• 6050 - prefill updateID in the search box for DLT pipeline updates when navigating from alert to reports view

❤️ Health Diagnostics // Performance

• 4342 - Add alerts for jobs configured with all purpose clusters and failed jobs

• Overprovisioning and Underprovisioning alerts

• 5808 - enhance design for performance alerts to make information more visible

• 6046 - add pipeline name and update id to DLT health alerts tickets

• 6029 - set default sorting to DESC by Event time on Health Alerts

📙Performance Evaluation Report

• 6115 - waste and latency score

• 6077 - added orchestration score

• 6097 - heavy driver and light cluster workload detection

📊 Analysis Optimizations

• [5979] - add "Reanalyze Notebooks Telemetry" and "Reanalyze DLT Pipeline Analysis" buttons

• [5214] - support reanalyzing telemetry for a job run that is already finished

🪩 UX Optimizations

• [5737] - uniquely identify each selected tab on Settings based on the URL

Operational Management // Telemetry Analysis

• 4807 - add tabs for DLT maintenance jobs

  • enhanced analysis history for multiple analysis types

• 6159 - Update cluster events analyzer. If the cluster events analysis fails for a certain cluster, the analysis should be marked as finished with warnings

📈 Operational Management // Consumption (Cost) Analysis

• 6041 - Proportionally compute DLT update costs

📙 AWS Support

• 5550: expose Service Principal token expire policy

Miscellaneous

• 5803 - Enhance license key protection from bad user input

🐞 Fixed Bugs

• 5267 - [aws] add creator when missing for pipeline entry on DLT provisioning page

🎁 Coming Soon

Pool contention report

Subscription cost breakdown