Prerequisite: Follow instructions provided to setup lakehouse infrastructure resources

TODO: provide link to confluence page with detailed infrastructure requirements and setup

...

Step 1) Required Resources

Lakehouse Monitor requires the following resources to already be created:

• TODO link to aws resource requirements

Step 2) Configuration Prerequisites

• AWS Secrets Manager needs to be configured with the following secret key value pairs. Suggested name for the secret is ‘bplm-credentials’:

  • storage-access-key - DynamoDB access key

  • storage-secret-key - DynamoDB secret key

  • service-account-username - Databricks service account username

  • service-account-password - Databricks service account password

  • mssql-password - SQL admin password

  • application-encryption-secret -

Step 3) Installation procedure

1. SSH into the BPLM VM configured at Step 1) Required Resources.

• Download the install archive by running the following command:
  TDODO UPDATE WITH LATEST LINKwget https://bplmdemoappstg.blob.core.windows.net/deployment/vm-aws/archive.zip

• Extract the archive contents
  unzip archive.zip

In the destination directory you should see the following files:

  .env
  docker-compose.yml
  setup.sh
  start.sh
  

1. Before you start setup you need to fill out the .env file with the required information). Open the file in your editor of choice and fill in the values.

   1. Please find a brief explanation of the .env values below

LOG_LEVEL=info
LOG_LEVEL_APP=info
LOG_LEVEL_HTTP_HEADERS=error

APPSERVICE_URL=<eg:https://demo.aws-bplm.com> 

SQL_DATABASE=master
SQL_SERVER_HOST=<eg:192.168.4.10>
SQL_USER=<eg:sql_admin>

STORAGE_AWS_REGION=<eg:us-west-1>
STORAGE_AWS_TABLE_PREFIX=bplm

AWS_SECRETS_MANAGER_ENABLED=true
AWS_SECRETS_MANAGER_REGION=<eg:us-west-1>

SERVER_SSL_ENABLED=true
SERVER_SSL_KEY-STORE=/keystore/bplm.p12
SERVER_SSL_KEY-STORE-PASSWORD=
SERVER_SSL_KEY-STORE-TYPE=PKCS12
SERVER_SSL_KEY-ALIAS=bplm
SERVER_SSL_KEY-PASSWORD=

SERVICE_PRINCIPAL_CLIENTID=<eg: 925accb1-8506-4ec4-a90b-b1b0e6d8a5eb>
SERVICE_PRINCIPAL_TENANTID=<eg: 03786a4c-412b-4fac-a981-b4c5bcbc55b7>
SERVICE_PRINCIPAL_CLIENTSECRET=<secret value>

DATABRICKS_ACCOUNT_ID=<eg: 56293882-89e7-4ecd-a5f7-cb61e68a54f0>
DATARICKS_SERVICE_PRINCIPAL=<eg: 48de6ad6-ff14-403d-b842-d4ce5da4662f>
ACTIVE-DIRECTORY_HOST=https://login.microsoftonline.com
ACTIVE-DIRECTORY_TOKEN-ENDPOINT=/oauth2/v2.0/token
ACTIVE-DIRECTORY_AUTHORIZE-ENDPOINT=/oauth2/v2.0/authorize
ACTIVE-DIRECTORY_JWK-ENDPOINT=/discovery/keys
ACTIVE-DIRECTORY_USER-INFO-URI=https://graph.microsoft.com/oidc/userinfo

CLOUD_PROVIDER=AWS
AUTHENTICATION_PROVIDER=databricks-account,active-directory
SPRING_PROFILES_ACTIVE=production-aws
SERVER_SERVLET_SESSION_PERSISTENT=true
SERVER_SERVLET_SESSION_STORE_DIR=<eg: /home/localuser/dockerless-env/spring-session/session>
ADMIN_APP_ROLE=internal_user
METRIC_PROCESSING_ENABLED=false
#metric.queueMonitoring.compactionTimeout=PT25M

2. In the same region as the rest of the resources, create a secret in Secret Manager with the name 'bplm-credentials' and add the key value pairs below ( example given as plain text JSON entry )

{
   
"storage-access-key":"<>",
	"storage-secret-key":"<>",
	"service-account-username":"<databricks service account name>",
	"service-account-password":"<>",
	"mssql-password":"<sql admin password>",
	"application-encryption-secret":"<>"
}

3. Copy setup scripts and completed .env file template onto the host VM

docker-compose.yml, .env, setup.sh, and start.sh

...

4. Run ./setup.sh providing the domain you wish to create an SSL cert for, the version of the lakehouse monitor, and an admin email that will be used to configure certbot’s notifications when creating an SSL certificate.

...