Table of Contents | ||||||
---|---|---|---|---|---|---|
|
1. Backing up
In this section you will find the process for backing up the MSSQL DB that is running inside a docker container for the Lakehouse MonitorOptimizer.
1. Stop the containers
First and foremost you’ll need to stop the app and the database:
...
This tells you that the Lakehouse monitor Optimizer and the DB are down.
2. Locate the MSSQL data
Now you will need to identify the path on the host system where your MSSQL data resides. Do this by using the following approach:
Code Block |
---|
$ sudo -i # docker volume ls DRIVER VOLUME NAME local centos_mssql_data local centos_spring_session # docker volume inspect centos_mssql_data [ { "CreatedAt": "2023-03-15T07:27:39Z", "Driver": "local", "Labels": { "com.docker.compose.project": "centos", "com.docker.compose.version": "2.16.0", "com.docker.compose.volume": "mssql_data" }, "Mountpoint": "/var/lib/docker/volumes/centos_mssql_data/_data", "Name": "centos_mssql_data", "Options": null, "Scope": "local" } ] # cd /var/lib/docker/volumes/centos_mssql_data/_data # ls -la total 8 drwxrwx---. 7 root root 73 Apr 5 12:36 . drwx-----x. 3 root root 19 Mar 15 07:27 .. drwxr-xr-x. 5 10001 root 74 Mar 15 07:27 .system drwxr-xr-x. 2 10001 root 24 Apr 5 12:36 backup drwxr-xr-x. 2 10001 root 4096 Mar 15 07:27 data drwxr-xr-x. 2 10001 root 4096 Apr 6 10:10 log drwxr-xr-x. 2 10001 root 25 Mar 15 07:27 secrets |
3. Archive the data
Once in this step, you can create a backup archive of this entire directory using the tar
command
...
Code Block |
---|
# cd ~centos # chown centos mssql-backup.tgz |
4. Start the containers
You can now leave the privileged session using the exit
command and restart the Lakehouse containers
...
Code Block |
---|
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 240a0312e897 blueprint.azurecr.io/bpcs/lakehouse-optimizer:20230406.1 "./start-app.sh" 4 seconds ago Up 3 seconds 2222/tcp, 0.0.0.0:4040->4040/tcp, :::4040->4040/tcp, 0.0.0.0:80->8080/tcp, :::80->8080/tcp bplm 0287182a2d92 mcr.microsoft.com/mssql/server:2019-latest "/opt/mssql/bin/perm…" 4 seconds ago Up 3 seconds 0.0.0.0:1433->1433/tcp, :::1433->1433/tcp lakehouse |
5. Use scp
to download the data
You can use scp
to copy the tgz
backup to your local machine and from there upload it to S3 or any other cloud storage solution you use.
...
This will copy the archive to my local computer from which point I can upload it to S3.
2. Restore
In this section you will find the process for using the backup created in the first section to restore the MSSQL DB that’s running inside a docker container for the Lakehouse MonitorOptimizer.
Assumptions
I’m assuming that you have the mssql-backup.tgz
file on your server under the centos
user home folder, if not, you need to use scp
to copy it from your local machine, so on your local machine you run this:
Code Block |
---|
$ scp <path to tar>/mssql-backup.tgz <your username>@<your server host>: |
1. Stop the containers
First and foremost you’ll need to stop the app and the database:
...
This tells you that the Lakehouse monitor Optimizer and the DB are down.
2. Locate the MSSQL data
Now you will need to identify the path on the host system where your MSSQL data resides. Do this by using the following approach:
Code Block |
---|
$ sudo -i # docker volume ls DRIVER VOLUME NAME local centos_mssql_data local centos_spring_session # docker volume inspect centos_mssql_data [ { "CreatedAt": "2023-03-15T07:27:39Z", "Driver": "local", "Labels": { "com.docker.compose.project": "centos", "com.docker.compose.version": "2.16.0", "com.docker.compose.volume": "mssql_data" }, "Mountpoint": "/var/lib/docker/volumes/centos_mssql_data/_data", "Name": "centos_mssql_data", "Options": null, "Scope": "local" } ] # cd /var/lib/docker/volumes/centos_mssql_data/_data # ls -la total 8 drwxrwx---. 7 root root 73 Apr 5 12:36 . drwx-----x. 3 root root 19 Mar 15 07:27 .. drwxr-xr-x. 5 10001 root 74 Mar 15 07:27 .system drwxr-xr-x. 2 10001 root 24 Apr 5 12:36 backup drwxr-xr-x. 2 10001 root 4096 Mar 15 07:27 data drwxr-xr-x. 2 10001 root 4096 Apr 6 10:10 log drwxr-xr-x. 2 10001 root 25 Mar 15 07:27 secrets |
3. Clean the _data
dir
You can either remove all of the data or, for extra-precations move it into another directory of your choosing.
3.1 Deleting the data
Code Block |
---|
# rm -rfv * .system |
3.2 Moving the data
Code Block |
---|
# mkdir bak # mv * .system bak mv: cannot move 'bak' to a subdirectory of itself, 'bak/bak' # ls -la total 0 drwxrwx---. 3 root root 17 Apr 6 10:19 . drwx-----x. 3 root root 19 Mar 15 07:27 .. drwxr-xr-x. 7 root root 73 Apr 6 10:19 bak |
4. Restoring the data
With the destination directory empty, it is now time to bring in the backup data.
...
Code Block |
---|
# cd - /var/lib/docker/volumes/centos_mssql_data/_data # ls -la total 8 drwxrwx---. 8 root root 84 Apr 5 12:36 . drwx-----x. 3 root root 19 Mar 15 07:27 .. drwxr-xr-x. 5 10001 root 74 Mar 15 07:27 .system drwxr-xr-x. 2 10001 root 24 Apr 5 12:36 backup drwxr-xr-x. 7 root root 73 Apr 6 10:19 bak drwxr-xr-x. 2 10001 root 4096 Mar 15 07:27 data drwxr-xr-x. 2 10001 root 4096 Apr 6 10:10 log drwxr-xr-x. 2 10001 root 25 Mar 15 07:27 secrets |
5. Starting the containers
You can now exit the privileged bash session and start the containers.
...
Code Block |
---|
$ docker logs bplm | grep Started ..... 2023-04-06 10:21:08.024 INFO 1 --- [ main] scala.App : Started App in 34.412 seconds (JVM running for 36.438) |
6. Clean up of the bak
folder
If in step 3 you chose to create the bak
folder, it is now safe to delete it.
...