Events Service Data Migration Failure: Shard Count Mismatch

Events Service data migration fails with the following errors:

  1. Check the {HOME}/migration-tool/SERVICE-MIGRATION-STATUS.txt file to verify the Events Service data migration status.
    CODE 
    Migration Status - Updated: 2025-12-09 13:24:55
====================================================
EUM        | Status: running
SYNTH      | Status: running         
CONTROLLER | Status: saturated 
EVENTS     | Status: failed
====================================================
  2. If the Events Service data migration status is failed, in some logs, you may observe the following pattern:
    PYTHON 
    config_manager - restore_worker - ERROR - Elasticsearch-migration-restore-worker: Restore submit failed: ApiError(500, 'snapshot_restore_exception', '[appdynamics-analytics-backup:<snapshot_name>/<snapshot_id>] cannot restore index [<index_name>] with [<index_shard_count_1>] shards from a snapshot of index [<index_name>] with [<index_shard_count_2>] shards')
    Example log:
    PYTHON 
    config_manager - restore_worker - ERROR - Elasticsearch-migration-restore-worker: Restore submit failed: ApiError(500, 'snapshot_restore_exception', '[appdynamics-analytics-backup:snapshot_20260223_131558/smjrnOQhREWIsKX9R6U7Ag] cannot restore index [appdynamics_schemas] with [1] shards from a snapshot of index [appdynamics_schemas] with [3] shards')

This error pattern appear if there is a mismatch in the shard count between classic and VA for the same index names. As a result, the data migration for those indices fail.

Delete the problematic index on Virtual Appliance so that the migration tool can correctly sync the shard count for such indices during data migration.

  1. Log in to Virtual Appliance and run following command:
    CODE 
    appduser@appdva-test7-vm348:~$ kubectl get svc -n es
    Sample output:
    CODE 
    NAME                    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
appd-es-http            ClusterIP   10.152.183.210   <none>        9200/TCP   6d19h 
appd-es-internal-http   ClusterIP   10.152.183.56    <none>        9200/TCP   6d19h
appd-es-node            ClusterIP   None             <none>        9200/TCP   6d19h
appd-es-transport       ClusterIP   None             <none>        9300/TCP   6d19h
  2. Specify the cluster IP address of the <appd-es-http> service and delete the problematic index from the Virtual Appliance (VA).
    CODE 
    curl -u username:password -X DELETE "http://10.0.0.1:9200/<index_name>"
    Note: Obtain the username and password for this command from the config.yaml file. Go to the services.events section and copy the username and password.
    CODE 
    username: va_es_user
password: va_es_password