Product Features (AWS)
Blueprint Lakehouse Optimizer for Databricks features real time and predictive analysis of what your organization has spent by Subscription, Workspace, Resource and User, allocating usage to specific business functions, reports, and applications. This gives you full visibility into your telemetry and cost data, you will also be able to monitor your cloud expense. The complete solution includes starter kits with PowerBI Dashboard templates for custom reporting and Databricks ingestion notebooks for advanced cost and performance analysis. Continue reading to learn more about how Lakehouse Optimizer can help your organization.
Comprehensive Insights
Lakehouse Optimizer offers comprehensive overview of where and when costs are made. Users can easily customize data views by date ranges, cumulative or granular costs, Workspaces, and/or AWS versus Databricks costs. Monthly and daily changes are recognized and displayed via widgets so you can better understand your costs in comparison to previous spending.
Homepage with Executive Summaries
Provide a Budget Estimator widget to keep costs aligned with planned expenses.
View a breakdown of granular costs between cloud provider and Databricks for individual workspaces and subscriptions. (VMs, VM Disks, Network, Storage, Pools)
Detailed Cost and Performance Reporting
Cost and telemetry metrics are backed by a database to facilitate lightning-fast analytics and reporting.
Tracking past runs of jobs, including deleted jobs and background processes for true auditability of the environment.
All-Purpose Clusters
Provides extensive reporting with all Jobs and Notebooks that are run-on All-Purpose Clusters.
User Activity Reporting
Provides extensive reporting with a focus on the activity of users and usage patterns of computing resources.
Present stats per clusters with the activity of users
Show idleness distribution per user
Show activity distribution per user in a cluster
Delta Live Tables Monitoring
Enable monitoring and reporting with cost and telemetry information for your DLT pipelines.
Intelligent Recommendations
Non-Optimal Configuration Warnings
Warn on possible misuse of all-purpose-clusters when job-clusters can be used
See Clusters configuration panel when enabling and disabling the monitor agent
Advanced Cost Analysis
AWS Costs
Supports Databricks DBCU Reservations
Databricks Credit Units Reservations
A company buys in advanced DBU units (prepaid) units, the company receives a discount for the prepaid amount
Cost per job widget on reporting page supports discounted prices
Use amortized cost provided by AWS for scenarios with reserved (prepaid) DBCU
Support both pay-as-you-go scenarios and prepaid resources (reservations)
Support Consumption Data Loading for AWS via Databricks notebook by using the bpcs-consumption module
Load Consumption Data and store it in AWS Blob Storage
Databricks Pools Support
Security
Active Directory integration
Supports AWS managed identity
Use private endpoints across VMs to access cost and telemetry data saved in AWS storage
Modular deployment
Supports Databricks on AWS
Supports Databricks on AWS
Can be run in a VM with Active Directory integration
Distributed design
Large amounts of data can be analyzed by using modules deployed
What’s Next?
Have questions or would like a live demo? Please Contact Us for further discussion. We are here to help you make the most out of your Lakehouse Optimizer experience!