Events Service Data Restore Fails with a 500 Error

During the data migration of Events Service, the Elasticsearch restore fails with a 500 error.

  1. Check the {HOME}/migration-tool/SERVICE-MIGRATION-STATUS.txt file to verify the Events Service data migration status.
    CODE 
    Migration Status - Updated: 2025-12-09 13:24:55
====================================================
EUM        | Status: running
SYNTH      | Status: running         
CONTROLLER | Status: saturated 
EVENTS     | Status: failed
====================================================
  2. If the Events Service data migration status is failed, in some logs, you may observe the following pattern:
    PYTHON 
    config_manager - restore_worker - ERROR - Elasticsearch-migration-restore-worker: Restore submit failed: ApiError(500, 'repository_exception', '[appdynamics-analytics-backup] Could not determine repository generation from root blobs', /mnt/nfs_events/index-N: Operation not permitted)
    Example log:
    PYTHON 
    config_manager - restore_worker - ERROR - Elasticsearch-migration-restore-worker: Restore submit failed: ApiError(500, 'repository_exception', '[appdynamics-analytics-backup] Could not determine repository generation from root blobs', /mnt/nfs_events/index-4: Operation not permitted)

This 500 error occurs while restoring the Elasticsearch indices because of some transient errors like network or storage errors.

Increase the the retry and failure thresholds in the migration tool's configuration file.

  1. Log in to Virtual Appliance node.
  2. Go to /home/appduser/migration-tool.
    CODE 
    cd /home/appduser/migration-tool
  3. Increase self.max_retry_attempts and self.max_consecutive_failures to 30.
    Note: If the issue persists, you require to double the existing value for these parameters again, continue to increase the counts until the issue is fixed.
    CODE 
    sed -I \
  -e 's/self\.max_retry_attempts = 5/self.max_retry_attempts = 30/' \
  -e 's/self\.max_consecutive_failures = 3/self.max_consecutive_failures = 30/' \
  src/datastores/elasticsearch.py
  4. Restart the migration tool.