Back Up Using the S3 Compatible Storage of MinIO

You can back up the Virtual Appliance data to MinIO S3-compatible storage. Follow these steps to back up the Virtual Appliance data:

  1. Set up MinIO S3 Compatiable Storage
  2. Configure MinIO certificates on Virtual Appliance
  3. Configure the MinIO endpoint
  4. Back up the Virtual Appliace data
  5. Restore Virtual Appliance

Set Up MinIO S3 Compatible Storage

Before you back up the Virtual Appliance data using MinIO, follow these steps to set up the MinIO server:

Note: Ensure that the 9000 port is available for the MinIO server.
  1. Download the MinIO server for the Linux OS.
  2. Grant the execute permission to the MinIO file.
    chmod +x minio
  3. Create a directory to store CA certificates.
    mkdir -p ~/.minio/certs/CAs
cd ~/.minio/certs
  4. Generate Certificate Signing Request and private key for the MinIO server:
    openssl genpkey -algorithm RSA -out private.key
openssl req -new -key private.key -out cert.csr
openssl x509 -req -days 365 -in cert.csr -signkey private.key -out public.crt
./minio server --address :9000 /mnt/data

Configure MinIO Certificates on the Virtual Appliance

You require the CA certificates of MinIO on the Virtual Appliance to establish an SSL connection between the MinIO server and store snapshots.

Copy public.crt of MinIO to the Virtual Appliance cluster.

PostgreSQL
/var/appd/config/postgres-s3-ca.crt

MySQL
/var/appd/config/mysql-s3-ca.crt

Elasticsearch requires the JKS format so convert the public.crt to JKS file.

  1. Run the following command to generate the JKS file from the public.crt :
    keytool -importcert -keystore es-s3-ca.jks -storepass changeit -file public.crt -alias minio
  2. Copy the es-s3-ca.jks fileto the following location:
    /var/appd/config
  3. Configure the truststore password in the secrets.yaml . For example, changeit .See Edit the secrets.yaml.encrypted file.
    Note:
    • Access the MinIO Console to generate access and secret keys.
    • Specify the storepassword of es-s3-ca.jks as the trustStorePassword in secrets.
    • To sync the changes in secret.yaml, run the appdcli sync command:
    # S3 secrets
##
backup:
elasticsearch:
s3:
access_key: WTFaWld3c1NHWWNOSzBWeDlXaHI=
secret_key: TWtJY0NweVdIcEhJV0pxdzdIWkRnVld3dzAxODRqbjhnS1pnZHozag==
trustStorePassword: changeit

Configure the MinIO Endpoint in theglobals.yaml.gotmplFile

  1. Create a repository to back up the data in the S3 compatible storage.
    Note: You can create repository only for the Elasticsearch backup. By default, the globals.yaml.gotmpl file uses the repo2 repository for MySQL and PostgreSQL. 
    appdcli run es_backup create-s3-repo virtual-appliance-repo-minio
  2. Enable thes3 section for the datastores to back up using the MinIO server. See globals.yaml.gotmpl file.
    Elasticsearch
    # Backup config for datastores
backup:
elasticsearch:
s3:
enabled: true
endpoint: https://s3-endpoint
protocol: https
bucket: es-bucket
region: us-east-1
{{ if isFile "/var/appd/config/es-s3-ca.jks" }}
certFile: {{ readFile "/var/appd/config/es-s3-ca.jks" | b64enc | quote }}
{{ end }}
    PostgreSQL
    # Backup config for datastores
backup:
postgresql:
s3:
enabled: true
repoName: repo2
endpoint: https://10.0.203.142:9000
bucket: postgresql-bucket
region: us-east-1
{{ if isFile "/var/appd/config/postgres-s3-ca.crt" }}
certFile: {{ readFile "/var/appd/config/postgres-s3-ca.crt" | b64enc | quote }}
{{ end }}
    MySQL
    # Backup config for datastores
backup:
mysql:
s3:
enabled: true
repoName: repo2
endpoint: http://10.0.203.142:9000
bucket: mysql-bucket
region: us-east-1
{{ if isFile "/var/appd/config/mysql-s3-ca.crt" }}
certFile: {{ readFile "/var/appd/config/mysql-s3-ca.crt" | b64enc | quote }}
{{ end }}
    Note: Do not specify the HTTPS endpoint because MySQL currently supports only HTTP endpoint.

Back Up the Virtual Appliance Data in S3 Compatible Storage

Snapshots capture the state of the datastore and can be used to restore data. You can store multiple snapshots in a single repository. The following commands stores the snapshots in the virtual-appliance-repo-minio repository.

Create a snapshot for each datastore:

Elasticsearch
appdcli run es_backup create-snapshot <repo-name> <snapshot-name>
Example 
appdcli run es_backup create-snapshot virtual-appliance-repo-minio es-snapshot
PostgreSQL
appdcli run pg_backup create-snapshot <repo-name> <snapshot-name>
Example 
appdcli run pg_backup create-snapshot repo2 pg-snapshot
MySQL
appdcli run mysql_backup create-snapshot <repo-name> <snapshot-name>
Example 
appdcli run mysql_backup create-snapshot repo2 mysql-snapshot

Restore Snapshots from the MinIO server

Use the snapshots that you have stored in MinIO server to restore the datastores.

Note: You may face issues while restoring a large backups. In such cases, bring the Controller down.
helm delete controller -n cisco-controller
  1. List the snapshots in the repository:
    Elasticsearch
    appdcli run es_backup list-snapshot <repo-name>
    Example
    appdcli run es_backup list-snapshot virtual-appliance-repo-mino
    PostgreSQL
    appdcli run pg_backup list-snapshot <repo-name>
    Example
    appdcli run pg_backup list-snapshot repo2
    MySQL
    appdcli run mysql_backup list-snapshot <repo-name>
    Example
    appdcli run mysql_backup list-snapshot repo2
  2. Restore the datastore by one of the snapshots:
    Elasticsearch
    appdcli run es_backup restore-snapshot <repo-name> <snapshot-name>
    Example
    appdcli run es_backup restore-snapshot virtual-appliance-repo-mino es_snapshot
    PostgreSQL
    appdcli run pg_backup restore-snapshot <repo-name>
    Example
    appdcli run pg_backup restore-snapshot repo2 pg_snapshot
    MySQL
    appdcli run mysql_backup restore-snapshot <repo-name>
    Example
    appdcli run mysql_backup restore-snapshot repo2 mysql_snapshot

After the restore is successful, if the Controller is down, run the following command:

appdcli sync appd <profile>

Delete the Snapshots from the MinIO S3 Compatible Storage

If you no longer require the snapshot, you can delete the snapshot from the MinIO S3 Compatible Storage.

Elasticsearch
appdcli run es_backup delete-snapshot <repo-name> <snapshot-name>
Example 
appdcli run es_backup delete-snapshot virtual-appliance-repo-mino es_snapshot
PostgreSQL
appdcli run pg_backup delete-snapshot <repo-name>
Example 
appdcli run pg_backup delete-snapshot repo2 pg_snapshot
MySQL
appdcli run mysql_backup delete-snapshot <repo-name>
Example 
appdcli run mysql_backup delete-snapshot repo2 mysql_snapshot