Send alert notifications to BigPanda using Splunk Observability Cloud

Configure Splunk Observability Cloud to send alerts to Amazon BigPanda when a detector alert condition is met and when the condition clears.

You can configure Splunk Observability Cloud to automatically send alert notifications to BigPanda when a detector alert condition is met and when the alert clears.

To send Splunk Observability Cloud alert notifications to BigPanda, complete the following configuration tasks:

Step 1: Create a Splunk Observability Cloud integration in BigPanda

You must be a BigPanda administrator to complete this task.

To create a Splunk Observability Cloud integration in BigPanda:

  1. Log in to BigPanda.

  2. Access the Integrations page and select New Integration.

  3. Hover over the ALERTS REST API tile and select Integrate.

  4. Enter a descriptive name for the integration and select Generate App Key.

  5. The app key displays. Copy the app key for use in Step 2: Create a BigPanda integration in Splunk Observability Cloud.

  6. Select >ALERTS REST API. Copy the bearer token for use in Step 2: Create a BigPanda integration in Splunk Observability Cloud.

Step 2: Create a BigPanda integration in Splunk Observability Cloud

You must be a Splunk Observability Cloud administrator to complete this task.

To create a BigPanda integration in Splunk Observability Cloud:

  1. Log in to Splunk Observability Cloud.

  2. Open the BigPanda guided setup. Optionally, you can navigate to the guided setup on your own:

    1. In the left navigation menu, select Data Management.

    2. Go to the Available integrations tab, or select Add Integration in the Deployed integrations tab.

    3. In the integration filter menu, select All.

    4. In the Search field, search for BigPanda, and select it.

    5. Select +Add Integration to display the configuration options.

  3. Review the Summary info and select Next.

  4. In Configure integration, enter the following details:

    1. Name: Enter your integration a unique and descriptive name. For information about the downstream use of this name, see About naming your integrations.

    2. App key: Enter the app key you copied from BigPanda in Step 1: Create a Splunk Observability Cloud integration in BigPanda.

    3. Token: Enter the token you copied from BigPanda in Step 1: Create a Splunk Observability Cloud integration in BigPanda.

  5. Select Next. If Splunk Observability Cloud can validate the BigPanda app key and token, a Validated! success message displays. If you get an error, make sure that the app key and token values you entered match the values displayed in BigPanda in Step 1: Create a Splunk Observability Cloud integration in BigPanda.

  6. In Customize message, configure the payload that you want to send to BigPanda from Splunk Observability Cloud. You can choose whether to keep the default BigPanda payload or add properties that enrich BigPanda incidents with organization-specific context, such as service, environment, team, runbook, cluster, host, or other detector dimensions. You can customize the following messages:

    • Triggered message: defines the payload sent when a detector condition is met and a new BigPanda incident or alert event is created. This message should include the active alert state, severity, detector and rule details, impacted entity dimensions, and any routing or ownership fields required by BigPanda.

    • Resolved message: defines the payload sent when the alert clears, is manually resolved, or the detector stops. This message can include closure context and resolution metadata.

    Note: Both message templates must continue to produce valid BigPanda JSON payloads and retain required correlation fields. Keeping the same identifying properties across triggered and resolved messages is important so BigPanda can match the resolved event to the original triggered event.

  7. Review and save the the integration details.

  8. Use the Run test option to validate the payload configuration.

Run test

The Run test feature enables you to validate the BigPanda payload configuration after the triggered and resolved message templates have been saved.

To test the configuration:

  1. On the BigPanda integration details page, select Run test to open the Run BigPanda integration test dialog.

  2. In the dialog, select a log level for the test. Available log levels include Debug, Error, Warn, Info, and Trace.The dialog displays separate read-only output panels for the Triggered message and Resolved message.

  3. Select Run test to start the test using the selected log level and populate both output panels with the resulting payload data.

This test flow helps confirm that both triggered and resolved payloads are structurally valid and contain the expected content before the integration goes live.

Step 3: Add a BigPanda integration as a detector alert recipient in Splunk Observability Cloud

To add a BigPanda integration as a detector alert recipient in Splunk Observability Cloud:

  1. Create or edit a detector that you want to configure to send alert notifications using your BigPanda integration.

    For more information about working with detectors, see Create detectors to trigger alerts and Subscribe to alerts using the Detector menu.

  2. In the Alert recipients step, select Add Recipient.

  3. Select BigPanda and then select the name of the BigPanda integration you want to use to send alert notifications. This is the integration name you created in Step 2: Create a BigPanda integration in Splunk Observability Cloud.

  4. Activate and save the detector.

Splunk Observability Cloud sends an alert notification to BigPanda when the detector triggers an alert and when the alert clears.

In addition to sending a subject, description, and other information to BigPanda, the integration maps certain pieces of Splunk Observability Cloud detector information to corresponding BigPanda properties as described in the following table.

Splunk Observability Cloud information

BigPanda property and value

Alert severity is Critical

status: Critical

Alert severity is Major, Minor, Warning, or Informational

status: Warning

Alert clears or manually resolved, or detector stops

status: OK

Detector rule name

check: Detector rule name

Metric has a dimension named cluster

cluster: Value of the cluster dimension

Metric has a dimension named host

host: Value of the host dimension

Metric has any other dimension

Custom properties, each named sfx_<dimension-name>: Value of the dimension.

If there are any name collisions between Splunk Observability Cloud dimensions and BigPanda status or check properties, Splunk Observability Cloud creates a new custom property in BigPanda. For example, if there is a Splunk Observability Cloud dimension named status, Splunk Observability Cloud creates a custom property named sfx_status and stores the value of the status dimension there.