This policies are required by Lakehouse Optimizer App being deployed on an AWS EC2 instance running on same AWS account where monitored Databricks Workspaces belong to. |
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BplmAWSCostReader",
"Effect": "Allow",
"Action": "ce:GetCostAndUsage",
"Resource": "*"
}
]
}
|
Allow Read of created secret
You will need to update this policy definition with your aws account ID and secret name. |
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BplmSecretsReader",
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "arn:aws:secretsmanager:*:{AWS Account ID}:secret:{SecretNameHere}*"
}
]
}
|
Allow management of DynamoDB and Simple Queue Service
You will need to update this policy definition with your aws account ID. |
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BplmDynamoPolicy",
"Effect": "Allow",
"Action": [
"dynamodb:CreateTable",
"dynamodb:UpdateTable",
"dynamodb:UpdateTimeToLive",
"dynamodb:DescribeTable",
"dynamodb:Scan",
"dynamodb:Query",
"dynamodb:BatchGetItem"
],
"Resource": [
"arn:aws:dynamodb:*:{AWS Account ID}:table/*bplm*"
]
},
{
"Sid": "BplmSqsPolicy",
"Effect": "Allow",
"Action": [
"sqs:DeleteMessage",
"sqs:GetQueueUrl",
"sqs:ReceiveMessage",
"sqs:CreateQueue"
],
"Resource": [
"arn:aws:sqs:*:{AWS Account ID}:*bplm*"
]
}
]
}
|
Allow s3 bucket tag get\set used for reporting workspace storage cost
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BplmS3TaggingWkspStorageCostPolicy",
"Effect": "Allow",
"Action": [
"s3:GetBucketTagging",
"s3:PutBucketTagging"
],
"Resource": "arn:aws:s3:::*"
}
]
}
|
Allow EC2 tag management for reporting workspace Nat Gateway costs
You will need to update this policy definition with your aws account ID. |
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BplmNatGatewayTaggingPolicy",
"Effect": "Allow",
"Action": [
"ec2:DeleteTags",
"ec2:CreateTags"
],
"Resource": [
"arn:aws:ec2:*:{AWS Account ID}:natgateway/*",
"arn:aws:ec2:*:{AWS Account ID}:natgateway/*",
"arn:aws:ec2:*:{AWS Account ID}:vpc/*",
"arn:aws:ec2:*:{AWS Account ID}:internet-gateway/*",
"arn:aws:ec2:*:{AWS Account ID}:elastic-ip/*",
"arn:aws:ec2:*:{AWS Account ID}:route-table/*",
"arn:aws:ec2:*:{AWS Account ID}:vpc-endpoint/*"
]
},
{
"Sid": "BplmNatGatewayVpcsTaggingPolicy",
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"ec2:DescribeNatGateways",
"ec2:DescribeVpcEndpoints",
"ec2:DescribeAddresses",
"ec2:DescribeInternetGateways",
"ec2:DescribeRouteTables"
],
"Resource": "*"
}
]
} |
Delivered billable usage logs:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BplmBillableUsageS3ReadFiles",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<bucket>/<path_prefix>/*"
},
{
"Sid": "BplmBillableUsageS3ListBuckets"
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<bucket>",
"Condition": {
"StringLike": {
"s3:prefix": "<path_prefix>/*"
}
}
},
{
"Sid": "BplmDecryptS3WithKMS",
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "*"
}
]
}
|
One could use "Resource": "arn:aws:kms:{KMS key region}:{AWS Account ID}:key/{SSE KMS key for s3 bucket}" to narrow down the BplmDecryptS3WithKMS permission only for specific key |
CostAndUsage Report 1.0 for cloud costs:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CostAndUsageS3GetObject",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<bucket>/<path_prefix>/*"
},
{
"Sid": "CostAndUsageS3ListBucket"
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<bucket>",
"Condition": {
"StringLike": {
"s3:prefix": "<path_prefix>/*"
}
}
}
]
}
|
This is just aggregated version from the aforementioned templates |
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BplmAWSCostReader",
"Effect": "Allow",
"Action": "ce:GetCostAndUsage",
"Resource": "*"
},
{
"Sid": "BplmSecretsReader",
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "arn:aws:secretsmanager:*:{AWS Account ID}:secret:{SecretNameHere}*"
},
{
"Sid": "BplmDynamoPolicy",
"Effect": "Allow",
"Action": [
"dynamodb:CreateTable",
"dynamodb:UpdateTable",
"dynamodb:UpdateTimeToLive",
"dynamodb:DescribeTable",
"dynamodb:Scan",
"dynamodb:Query"
],
"Resource": [
"arn:aws:dynamodb:*:{AWS Account ID}:table/*bplm*"
]
},
{
"Sid": "BplmSqsPolicy",
"Effect": "Allow",
"Action": [
"sqs:DeleteMessage",
"sqs:GetQueueUrl",
"sqs:ReceiveMessage",
"sqs:CreateQueue"
],
"Resource": [
"arn:aws:sqs:*:{AWS Account ID}:*bplm*"
]
},
{
"Sid": "BplmS3TaggingWkspStorageCostPolicy",
"Effect": "Allow",
"Action": [
"s3:GetBucketTagging",
"s3:PutBucketTagging"
],
"Resource": "arn:aws:s3:::*"
},
{
"Sid": "BplmNatGatewayTaggingPolicy",
"Effect": "Allow",
"Action": [
"ec2:DeleteTags",
"ec2:CreateTags"
],
"Resource": "arn:aws:ec2:*:{AWS Account ID}:natgateway/*"
},
{
"Sid": "BplmNatGatewayVpcsTaggingPolicy",
"Effect": "Allow",
"Action": [
"ec2:DescribeVpcs",
"ec2:DescribeNatGateways"
],
"Resource": "*"
},
{
"Sid": "BplmBillableUsageS3ReadFiles",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<bucket>/<path_prefix>/*"
},
{
"Sid": "BplmBillableUsageS3ListBuckets",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<bucket>",
"Condition": {
"StringLike": {
"s3:prefix": "<path_prefix>/*"
}
}
},
{
"Sid": "BplmDecryptS3WithKMS",
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "*"
},
{
"Sid": "CostAndUsageS3GetObject",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<bucket>/<path_prefix>/*"
},
{
"Sid": "CostAndUsageS3ListBucket"
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<bucket>",
"Condition": {
"StringLike": {
"s3:prefix": "<path_prefix>/*"
}
}
}
]
} |
LHO Agent permission policy
Next permissions should be granted to the LHO Agent via IAM User, Instance Profile or Role assumption:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "WriteToDynamoDbAndSqs",
"Effect": "Allow",
"Action": [
"dynamodb:BatchWriteItem",
"dynamodb:PutItem",
"dynamodb:UpdateItem",
"sqs:SendMessage"
],
"Resource": [
"arn:aws:sqs:<optional_region_or_*>:{AWS Account ID}:bplm*",
"arn:aws:dynamodb:<optional_region_or_*>:{AWS Account ID}:table/bplm*"
]
}
]
} |