Upgrade the Virtual Appliance in KVM

Upgrading the Virtual Appliance involves:

  • Backing up the hard disks of the existing virtual machines.
  • Deploying new virtual machines by using QCOW2 image.
  • Restoring the backup to their hard disks.
  • Attaching the new hard disks to the older virtual machines

Splunk AppDynamics On-Premises Virtual Appliance provides the reference script that helps in upgrading the Virtual Appliance.

Follow the steps to upgrade the Virtual Appliance in KVM.

Back up the Virtual Appliance Data

Before you begin the upgrade, complete the following steps:

  1. Download the Virtual Appliance upgrade QCOW2 file from the Downloads portal.
  2. Shut down the Virtual Appliance:
    appdcli stop secapp
appdcli stop aiops
appdcli stop appd
<<Wait for pods to terminate before deleting operators>>
appdcli stop operators
  3. Run the backup command:
    appdcli run backup
    This command generates a <backup_tar> file with all the necessary data.
  4. Copy and paste the backup file outside of the cluster.

Upgrade the Virtual Appliance

  1. Download the upgrade script from the Splunk AppDynamics GitHub repository.
  2. Run the following command by specifying the locations  of the QCOW2 file and config.cfg .
    ./upgrade-cluster ../<filename>.qcow2 config.cfg

This script verifies the virsh

  1. Stops the virtual machine.

  2. Uploads the new QCOW2 image to the node storage pool.

  3. Deletes the OS disk image. However, it retains the data disk image.

  4. Creates a new OS disk from the template image.

  5. Starts the virtual machine.

Verify the Deployment Status

Verify the deployment of virtual machines:

  1. Verify whether the deployment of virtual machines is successful:
    appdctl show boot
    Note: Ensure that the services in each node is active. If any service appears as Failed, restart that virtual machine. You might have to redeploy the virtual machine if it is still failing.
  2. Create a three-node cluster:
    1. Run the following command in the primary node and specify the IP address of the peer nodes:
      appdctl cluster init <Node-2-IP> <Node-3-IP>
    2. Run the following command to verify the node status:
      appdctl show cluster
microk8s status
      Ensure that the output displays the Running status as true for the nodes that are part of the cluster.

      Sample Output

      NODE           | ROLE  | RUNNING
----------------+-------+---------
10.0.0.1:19001 | voter | true
10.0.0.2:19001 | voter | true
10.0.0.3:19001 | voter | true
      Note: You must re-login to the terminal if the following error appears:
      Insufficient Permissions to Access Microk8s

Restore Data in the Virtual Appliance

  1. Copy the <backup_tar> file to one of the cluster nodes that is generated earlier. See Prerequisites.
  2. Run the following command to restore the persistent volumes specifications:
    appdcli run restore <backup_tar>
  3. Verify that the PVC are in the Bound state by running the following command:
    kubectl get pvc -A
  4. Start the following services:
    appdcli start appd [Profile]
    1. Splunk AppDynamics Services.
    2. Anomaly Detection Service.
    3. Secure Application Service.
  5. Restore self-signed CA ingress certificates.A prompt is displayed "Do you want to restore self-signed ingress certificates?" Enter 0 for No or 1 for Yes".
    • Choose 0 if you have configured the certificates for DNS.
    • Choose 1 to configure the certificates.
  6. Sometimes, MySQL Router pods might fail to start. In such cases, restart the services using the following commands:
    1. Stop the Splunk AppDynamics services.
      appdcli stop appd

      Wait for the pods to terminate.

    2. Stop the operators:
      appdcli stop opeartors
    3. Start the Splunk AppDynamics services:
      appdcli start appd <profile>
After the upgrade, the cluster uses the existing data disk and becomes functional.