Run dual-platform validation

Use parallel Splunk AppDynamics and Splunk Observability Cloud monitoring to identify remaining differences before cutover.

Overview

Operate Splunk AppDynamics and Splunk Observability Cloud in parallel for the period approved in the migration plan. Use Splunk Observability Cloud for active monitoring and compare it with Splunk AppDynamics to identify remaining differences before cutover.

Prerequisites

Confirm that:

  • Complete controlled validation of migrated telemetry and configurations.
  • All in-scope applications continue sending data to both platforms.
  • No known critical data gap prevents a meaningful comparison.
  • Document the comparison period, success criteria, owners, and escalation path.
  • Operations teams and stakeholders know when the validation period begins and ends.
  • Production notification routing does not create uncontrolled duplicate paging.

Maintain an alert comparison log

For every critical alert observed during the comparison period, record:

Record Compare
Time Trigger time, evaluation delay, and clear time
Scope Application, service, environment, endpoint, or Business Transaction
Condition Metric, threshold, evaluation window, and condition grouping
Outcome Fired in both platforms, fired in one platform, or did not fire
Severity Splunk AppDynamics severity and Splunk Observability Cloud severity
Notification Recipient, delivery time, deduplication, and message content
Disposition Expected difference, configuration change, defect, or accepted gap

Investigate:

  • Missed critical alerts.
  • False positives.
  • Material timing differences.
  • Incorrect severity or affected entity.
  • Alerts that do not clear as intended.
  • Duplicate or missing notifications.

Tune detector thresholds or logic only after confirming that the two platforms evaluate comparable telemetry populations.

Compare dashboards over time

Compare corresponding dashboards using the same application activity, filters, and time ranges.

Include the time ranges required by the migration plan, such as:

  • 15 minutes for current behavior.
  • 1 hour for short-term trends.
  • 24 hours for daily patterns.
  • 7 days for weekly patterns.

For key metrics, compare:

  • Request rate or throughput.
  • Average response time.
  • Error rate.
  • Minimum and maximum values when operationally relevant.
  • P50, P90, P95, and P99 latency.
  • Groupings by service, environment, workflow, endpoint, host, or instance.

Confirm that the source and target use comparable units, rollups, filters, and request populations before treating a difference as a defect.

Observe telemetry and topology

Throughout the approved period:

  • Confirm that every required service and Business Transaction remains visible.
  • Review traces for representative normal, slow, and failed transactions.
  • Verify that Service Map relationships remain stable under normal and peak traffic.
  • Monitor for missing dimensions, intermittent export failures, or unexpected cardinality.
  • Confirm that data remains current after deployments, restarts, and scaling events.
  • Review agent and Collector logs when telemetry becomes incomplete or delayed.

Assess the comparison

Calculate and document the measures defined in the migration plan, including:

  • Critical-alert match rate.
  • Missed critical incidents.
  • False-positive rate.
  • Dashboard variance for key metrics.
  • Outstanding data, alert, feature, and usability gaps.

The playbook proposes at least 95 percent matching for critical alerts, no missed critical production incidents, less than 10 percent false positives, and less than 5 percent dashboard variance for key metrics. Treat these as planning defaults only. Use the criteria approved for the migration wave.

Complete the validation period

Complete dual-platform validation when:

  • The approved observation period has elapsed.
  • Alert and dashboard results meet the approved criteria.
  • Required services and Business Transactions remain visible.
  • No unexplained critical data or alert gap remains.
  • Every discrepancy has a documented disposition and owner.
  • The migration owner approves progression to operational-readiness and final gap-closure activities.