Cloud Costs via AWS CostExplorer

AWS CostExplorer

LHM Application IAM Role will assume an IAM Role in the Databricks Workspace AWS Account with a permission policy to access Cost Explorer data in that AWS Account Id:

IAM_Role_Cost_Explorer permission policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowGetCostAndUsages",
            "Effect": "Allow",
            "Action": "ce:GetCostAndUsage",
            "Resource": "*"
        }
    ]
}

Trust policy for the IAM_Role_Cost_Explorer that allows the LHO Application IAM Role in the app-host AWS Account to assume the cost explorer role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<LHM_App_Host_AWS_Account_ID>:role/<LHM_App_IAM_Role>"
            },
            "Action": "sts:AssumeRole",
        }
    ]
}

Permissions for cost-tagging NAT and S3

In order to obtain workspace storage and network costs, the LHO should be provided permissions to tag S3 bucket and NAT Gateway associated with the DBX

IAM_Role_Cost_Tag_For_S3_And_NAT permission policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "BplmS3TaggingWkspStorageCostPolicy",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketTagging",
                "s3:PutBucketTagging"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Sid": "BplmNatGatewayTaggingPolicy",
            "Effect": "Allow",
            "Action": [
                "ec2:DeleteTags",
                "ec2:CreateTags"
            ],
            "Resource": "arn:aws:ec2:*:<Databricks_Wksp_AWS_Account_Id>:natgateway/*"
        },
        {
            "Sid": "BplmNatGatewayVpcsTaggingPolicy",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeVpcs",
                "ec2:DescribeNatGateways"
            ],
            "Resource": "*"
        }
    ]
}

Trust policy for the IAM_Role_Cost_Tag_For_S3_And_NAT that allows the LHO Application IAM Role in the app-host AWS Account to assume the role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<LHM_App_Host_AWS_Account_ID>:role/<LHM_App_IAM_Role>"
            },
            "Action": "sts:AssumeRole",
        }
    ]
}

LHO application IAM Role permission policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AssumeCostExplorerRole",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::<Databricks_Wksp_AWS_Account_Id>:role/<IAM_Role_Cost_Explorer>"
        },
        {
            "Sid": "AssumeS3AndNatTagRole",
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::<Databricks_Wksp_AWS_Account_Id>:role/<IAM_Role_Cost_Tag_For_S3_And_NAT>"
        }
    ]
}

And in the Lakehouse Monitor .env config file provide the source role name for cost explorer and tags.

CROSS_ACCOUNT_ASSUME_IAM_ROLE_COST_EXPLORER_APP=arn:aws:iam::<Databricks_Wksp_AWS_Account_Id>:role/<IAM_Role_Cost_Explorer>
CROSS_ACCOUNT_ASSUME_IAM_ROLE_TAG_WORKSPACE_RESOURCE_APP=arn:aws:iam::<Databricks_Wksp_AWS_Account_Id>:role/<IAM_Role_Cost_Tag_For_S3_And_NAT>

In case of multiple AWS accounts hosting cloud resources for Databricks workspaces, you need to provide a role for each of them comma-separated.