Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Shutdown Timeout above Threshold

This policy specifies the minimum value which should be set for auto termination of a pool.

If the value is below the threshold, an incident will be generated.

Use this incident policy to generate incidents in order to minimize the amount of idling resources.

Databricks' pool configuration allows the user to specify an inactivity period in minutes after which the pool will be terminated automatically.

Databricks' allowed pool auto-termination interval must be between 1 and 10000 minutes.

Set the Idle Instance Auto Termination time to provide a buffer between when the instance is released from the cluster and when it’s dropped from the pool. Set this to a period that allows you to minimize cost while ensuring the availability of instances for scheduled jobs.

For example, job A is scheduled to run at 8:00 AM and takes 40 minutes to complete. Job B is scheduled to run at 9:00 AM and takes 30 minutes to complete. Set the Idle Instance Auto Termination value to 20 minutes to ensure that instances returned to the pool when job A completes are available when job B starts. Unless they are claimed by another cluster, those instances are terminated 20 minutes after job B ends.

Learn more:

  • No labels

0 Comments

You are not logged in. Any changes you make will be marked as anonymous. You may want to Log In if you already have an account.