Administer the Hybrid Virtual Appliance Deployment

Perform the following steps to administer the Virtual Appliance nodes in your Hybrid Deployment:

Disable the AIOps Service

If you are using Virtual Appliance Hybrid Deployment, you require to disable the AIOps service in your Controller. Follow the steps to disable Anomaly Detection service in Administration Console:
  1. Log in to the admin.jsp page and set the below properties to false:

    • anomaly.detection.enabled

    • anomaly.detection.deployed.onprem

  2. Click Save.
After setting these properties to false, the Anomaly Detection service in the Controller is temporarily disabled. This action interrupts communication between the Controller and the Virtual Appliance.

Stop the Virtual Appliance Services

Stop appd services on the Virtual Appliance.

Complete the following steps to stop the Virtual Appliance services:
  1. Stop the Anomaly Detection, Secure Application, ATD, and OpenTelemetry services.
    CODE 
    appdcli stop aiops|secapp|atd|otis
  2. Run the following command to stop Virtual Appliance services:
    CODE 
    appdcli stop appd
    Note: Wait for all pods to terminate, excluding the following namespaces:
    • appd-container-registry
    • cert-manager
    • elastic-system
    • kafka-operator
    • kube-system
    • mysql-operator
    • pg-operator
  3. Run the following command to stop operators, then wait till all the pods are terminated:
    CODE 
    appdcli stop operators
  4. Run the following command on all the Virtual Appliance nodes:
    CODE 
    sudo microk8s stop

Start the Virtual Appliance Services

Start the appd services on the Virtual Appliance

Complete the following steps to start the Virtual Appliance services:
  1. Start all the virtual machines.
  2. Run the following command to start microk8s.
    CODE 
    sudo microk8s start
  3. Verify the status of virtual machines.
    CODE 
    appdctl show boot
    Note: Ensure that the status of the services in each node appears as Success.
    Sample Output:
    CODE 
    NAME             | STATUS    | ERROR
------------------+-----------+-------
 enable-time-sync | Succeeded | --
 firewall-setup   | Succeeded | --
 hostname         | Succeeded | --
 microk8s-setup   | Succeeded | --
 ssh-setup        | Succeeded | --
 storage-setup    | Succeeded | --
 cert-setup       | Succeeded | --
  4. Start the Virtual Appliance services.
    CODE 
    appdcli start appd <profile>
appdcli start aiops <profile>
appdcli start secapp <profile>
appdcli start atd <profile>
appdcli start otis <profile>

Enable the AIOps Service

After you successfully start the services, enable the Anomaly Detection service to restore the communication between the Controller and the Virtual Appliance.
  1. Log in to the admin.jsp page and set the below properties to true:

    • anomaly.detection.enabled

    • anomaly.detection.deployed.onprem

  2. Click Save.

Generate the Hybrid Configuration File

Generate the hybrid configuration file when custom certificates are not used for Ingress and Kafka clusters.

Generate the hybrid configuration file only if you are not using custom certificates for Ingress and Kafka clusters.

Ensure that you have the latest CA certificates obtained after installing services. If not, update the CA certificates and regenerate the hybrid configuration file after restarting the service.

  1. Run the script to connect the Kafka node ports with the standalone Controller:
    CODE 
    ~/appd-charts/utils$ bash prepare-hybrid.sh

    Sample output:

    CODE 
    Writing Kafka CA Cert
Writing Schema registry CA Cert
Writing AnomalyDetectionKafkaSSLProducer
Writing AnomalyDetectionKafkaConsumerSSL
Writing configure script
Creating archive of hybrid config
hybrid-config/
hybrid-config/kafka-ca.crt
hybrid-config/AnomalyDetectionKafkaConsumerSSL
hybrid-config/configure.sh
hybrid-config/schema-registry-ca.crt
hybrid-config/AnomalyDetectionKafkaSSLProducer
Completed

    This script generates the hybrid-config.tar file that includes the required configuration files.

  2. Copy the hybrid-config.tar file to your standalone Controller instance.
  3. Log in to the standalone Controller.
    1. Locate and extract the hybrid-config.tar file contents.
      CODE 
      tar -xvf hybrid-config.tar
    2. Run the configure.sh script to provision the Controller with certificates and secrets to connect to the Kubernetes cluster:

      The following script is extracted from the hybrid-config.tar file.

      CODE 
      $ ./configure.sh
      Output:
      CODE 
      Using controller home as /home/appdynamics/appdynamics and keystore password as changeit
Do you want to proceed? (y/n): y

Creating Kafka truststore
Certificate was added to keystore

Creating schema registry truststore
Certificate was added to keystore
Certificate was added to keystore

Writing obfuscated producer config
Writing obfuscated consumer config

Configure MySQL for SSL
Using controller ip as <IP>
Is this IP address correct? (y/n): y

DNS name should match the Virtual Appliance cluster hybrid dbHost configuration
Please enter the DNS name (default: 10.0.0.1.nip.io): <DNS_Name_of_Classic_On_Premises_Controller>

Using controller IP as 10.0.0.1
Updating certs at /home/appdynamics/appdynamics/platform/controller/db/data using CA key file ca-key.pem and cert file ca.pem
Certificate request self-signature ok
subject=CN = mysql
MySQL certificate generated successfully: server-cert.pem
Completed
    Note:

    Certificates in the Virtual Appliance expire after 60 days. To maintain connectivity, you must copy the updated certificates from the Virtual Appliance to your Splunk AppDynamics On-Premises Classic Controller. To do this, regenerate the hybrid configuration file on the Virtual Appliance and then copy it to your Standalone Controller.