Set up Dynatrace alerts in ITSI

Prerequisites

  • Dynatrace environment with admin access (Settings permissions)

  • Splunk HEC endpoint accessible from Dynatrace (port 8088 or 443)

  • Splunk HEC token with write access to the target index:

    • Name: Dynatrace Problems

    • Source type: dynatrace:problem

    • Index: your target index (for example, main)

Create an alerting profile in Dynatrace

An alerting profile specifies which events trigger the notification.

  1. In Dynatrace, navigate to Settings then Alerting then Alerting profiles.

  2. Select + Add alerting profile.

  3. Configure severity and problem type filters as needed.

  4. Select Save changes.

Dynatrace webhook setup

  1. Navigate to Settings then Alerting then Notifications.

  2. Select + Add notification.

  3. Select Webhook as the notification type.

  4. Fill in the values:
    Field Value
    Display name Send to Splunk
    Webhook URL https://<your-splunk-host>:8088/services/collector
    Accept any SSL certificate Enable if using a self-signed certificate
    Call webhook if new events merge into existing problems Enable
    Call webhook if problem is closed Enable
    Alerting profile Select the profile created in Step 2.
  5. Under Additional HTTP headers, add the following:
    Header name Value
    Authorization Splunk <YOUR_HEC_TOKEN>
    Content-Type application/json
  6. Paste the following into the Custom payload field:
    JSON 
    {
  "event": {ProblemDetailsJSONv2},
  "source": "dynatrace",
  "sourcetype": "dynatrace:problemV2",
}
    Note: {ProblemDetailsJSONv2} has no quotes around. Dynatrace expands this as a raw JSON object following the Problems V2 API structure. It includes evidenceDetails and impactAnalysis, but excludes recentComments.

  7. Select Send test notification to verify connectivity.

  8. Select Save changes.

Test Dynatrace alert

Trigger or wait for a problem in Dynatrace, then confirm that it appears in Splunk:

index=main sourcetype="dynatrace:problem" | head 10