...
Usages in S3
The S3 bucket storing the billable usage (Databricks consumption data, DBU charges) requires an S3 bucket policy that will specify the scope of access for the LHO Application:
...
| Code Block |
|---|
# Full AWS Account where LHMLHO App is hosted
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3ReadObject",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<LHM<LHO_App_Host_AWS_Account_Id>:root"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<bucket>/<path_prefix>/*"
},
{
"Sid": "S3ListBucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<LHM<LHO_App_Host_AWS_Account_Id>:root"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<bucket>",
"Condition": {
"StringLike": {
"s3:prefix": "<path_prefix>/*"
}
}
}
]
}
|
...
| Code Block |
|---|
# Exactly the IAM Role of the LHMLHO Application in the AWS Account hosting it
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3ReadObject",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<LHM<LHO_App_Host_AWS_Account_Id>:role/<LHM<LHO_App_IAM_Role>"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<bucket>/<path_prefix>/*"
},
{
"Sid": "S3ListBucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<LHM<LHO_App_Host_AWS_Account_Id>:role/<LHM<LHO_App_IAM_Role>"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<bucket>",
"Condition": {
"StringLike": {
"s3:prefix": "<path_prefix>/*"
}
}
}
]
}
|
...
| Code Block |
|---|
# Exactly the IAM Role of the LHM Application in the AWS Account hosting it
{
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
{
... the default statement for local trusting ...
},
{
"Sid": "Allow use of the key to LHMLHO App IAM Role ",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<LHM<LHO_App_Host_AWS_Account_Id>:role/<LHM<LHO_App_IAM_Role>"
},
"Action": "kms:Decrypt",
"Resource": "*"
}
]
}
|
LHM LHO Application IAM Role permission policy:
...
You need to create a IAM role in the same AWS account as the S3 bucket with the Databricks billable usage logs. This role requires the following permission policies with access to the S3 bucket and AWS managed KMS key.
| Code Block |
|---|
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3ReadObject",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<bucket>/<path_prefix>/*"
},
{
"Sid": "S3ListBucket",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<bucket>",
"Condition": {
"StringLike": {
"s3:prefix": "<path_prefix>/*"
}
}
},
{
"Sid": "DecryptKMSbucket",
"Action": "kms:Decrypt",
"Effect": "Allow",
"Resource": "<ARN_OF_AWS_MANAGED_KMS_KEY_IN_SAME_REGION_AS_BUCKET>"
}
]
}
|
|
Trusting policy for the S3 role (only trusting a remote role version, for account-id or PrincipalOrgId, see the examples above):
| Code Block |
|---|
# Exactly the IAM Role of the |
|
LHMLHO Application in the AWS Account hosting it
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam:: |
|
<LHM<LHO_App_Host_AWS_Account_Id>:role/ |
|
<LHM<LHO_App_IAM_Role>"
},
"Action": "sts:AssumeRole"
}
]
}
|
|
LHM LHO Application IAM Role permission policy:
| Code Block |
|---|
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::<s3_aws_account_id>:role/<s3_role_name>"
}
]
}
|
|
Configuring Lakehouse monitor Optimizer to read from s3:
| Code Block |
|---|
CONSUMPTION_BILLABLE_USAGE_PATH=s3a://<bucket>/<path_prefix>/billable-usage/csv
STORAGE_AWS_S3_REGION=<bucket_region>
CROSS_ACCOUNT_ASSUME_IAM_ROLE_S3_DBX_BILLING_APP=arn:aws:iam::<s3_aws_account_id>:role/<s3_dbx_billing_role_name>
|
|