What Auto-Pilot does

• provisions automatically previously monitored assets

• provisions automatically new assets with shared-compute policies

• assists manual provisioning for assets that are owned by others

• automatically upgrades Telemetry Agent on LHO version upgrade

Automatic Provisioning

At each LHO restart and every 1h schedule, the Auto Pilot does the following when

• if Global Script ENABLED

  • enables monitoring on all jobs that contain at least one task with a shared-compute cluster as execution engine

  • enables monitoring on all clusters that are using shared-compute as access policy

  • all other jobs and clusters maintain their monitor status based on global init script option

• if Global Script DISABLED

  • enables monitoring on all jobs and clusters that contain were previously monitored (with an older LHO configuration)

    • e.g. the user changes the Secret Scope name and affects the monitoring status. Auto-Pilot reenables monitoring in this scenario

Assisting User-Triggered Provisioning

Auto-Pilot also allows admin users to enable monitoring on All-Purpose-Clusters or Jobs running on All-Purpose-Clusters that are owned by other users.

In this scenario, when provisioning is triggered manually by the user, Auto-Pilot temporary changes the ownership of the job or cluster to the user that triggered manually the provisioning and after the configuration is updated to reference the LHO Telemetry Agent, the ownership is set back to the original owner.

Ownership change is also executed when Auto-Pilot is triggered at restart or on a schedule basis. In this scenario, the new temporary owner is set to the Service Principal used by LHO.

Automatic Telemetry Agent Update

When a new version of LHO is available, on upgrade the Auto Pilot will update the LHO Telemetry Agent in each published workspace for all assets that were previously monitored or are of shared-compute type.

Please see section Automatic Provisioning for more details.

How to enable/disable Auto-Pilot

• env variables

  • AUTO_PILOT_ENABLED = true/false

  • default: true

Unity Catalog Support

If you are using Unity Catalog, Auto-Pilot requires additional configuration.

Please follow the following steps:

• Provisioning with Unity Catalog Enabled (AWS)