Introduction
Consumption Data is made available on a daily basis via Azure Consumption API in Azure, or via client Consumption and Usage API in AWS, starting with the previous day and going backward in time. The Loader allows you to preload consumption data for extended periods of time in order to speed up and facilitate advanced reporting.
This operation is done individually for each subscription in Azure, or each Account ID in AWS. The user can select the subscription/account in the upper left corner of the page.
Loader Configuration
The admin user can configure when, and how often to preload consumption data into LHO.
Loader Settings
The loader consists of 3 settings:
Load Data Since: preload consumption data dating back to this date all the way forward to the current data
Load Data:
Incrementally: Load only missing data from the specified interval
Refresh: Existing Consumption Data for the configured interval will be deleted and loaded again from the source
Batch Size in Number of Days: how much data to load in a batch of data being imported - the larger the number the more taxing it would be on the system
Schedule when to preload consumption data
The first tab, labeled Run Now will run the set configuration once on-demand, when the user clicks “Load”, but by clicking +ADD SCHEDULE the user can set when to automatically run the process and how often
Settings for the loader schedule include:
Schedule Name: friendly name to recognize one schedule from another
Enabled: is this schedule active - if not enabled the data will not load as scheduled
Frequency: How often to run and at what time of day (defaults to midnight)
Cluster Event Status
Cluster events are loaded from Databricks and processed in order to provide historical views of cluster lifecycle such as autoscaling timelines or cluster configuration timelines. The user can set a separate date for how far back these events go
Consumption Loading History
This logs the history of all the times consumption data was preloaded in the past as an operation. The last run is detailed in the right-side panel.
By Clicking Run History the user can bring up a dialog that details the full history of all consumption data loading done in the past, along with statuses of all the operations done.
0 Comments