AWS Lakehouse Optimizer Deployment Readiness Checklist

Below is a list of items that you will need to be aware of prior to deploying the Lakehouse Optimizer (LHO) on AWS with the Blueprint Deployment team:

1. Decide which Databricks workspaces should be monitored.

• If all your workspaces are in the same AWS account and you plan to deploy LHO into that same account, you will be deploying the Single AWS account scenario.

• Otherwise, you are deploying the Cross-account AWS scenario.

2. Determine which AWS region the LHO infrastructure will be deployed into. Ideally the same region as your Databricks workspaces to avoid cross-region data transfer fees.

3. For your selected region, ensure there’s an available VPC slot as the installation create a new VPC.

4. Understand AWS infrastructure resource requirements:

• AWS Resource Requirements | Required AWS Resources

5. The IAM user account running the installation needs the following policies attached:

• Deployment and Quick Setup Guide: AWS | AWS Account Prerequisites

6. If you have Microsoft Entra ID (Azure Active Directory) as your identity provider, create an app registration in your tenant:

• Deployment and Quick Setup Guide: AWS | Azure Active Directory Single Sign On Prerequisites

7. Have the following values available:

• App registration client ID

• Azure AD tenant ID

• The client secret value

8. Create the Databricks service principal –

• Account Admin

• Workspace Admin for all planned monitor workspaces

• Create OATH client secret and store for later use during deployment

9. Determine identity authorization solution for the LHO agent running on Databricks Workspace compute resources.

• Options outlined here -- AWS Resource Requirements - Blueprint Lakehouse Optimizer Documentation - Confluence (atlassian.net)

• Depending on options taken:

  • Create IAM user, saving username, access key and secret for later use.

  • Gather all instance profile ARNs in use by all target workspaces for later use, or have root account ARN available

  • Expand all in-use instance profile IAM policies to include the permissions required for the LHO Agent.

10. Decide on a DNS name

• As part of deployment, certificates are automatically created and a DNS entry is added if the hosted zone is available.

• If the desired DNS name is not a part of the AWS account LHO is deployed into, have someone who can create a DNS in the provider of choice

11. Enable CostExplorer in AWS

• Sometime after the deployment completes, activate the documented user-defined tags

If you are ready to get started, you can follow the AWS deployment guides here: