Configure your Splunk Observability Cloud account to collect AWS Bedrock metrics

Learn how to configure your Splunk Observability Cloud account to collect AWS Bedrock metrics.

You can monitor the performance of Amazon Web Services (AWS) Bedrock applications by configuring your AWS Bedrock applications to send metrics to Splunk Observability Cloud. This solution creates a cloud connection in your Splunk Observability Cloud account that collects metrics from AWS CloudWatch.

Complete the following steps to collect metrics from and monitor your AWS Bedrock applications.

  1. Connect AWS to Splunk Observability Cloud. For more information on the connection methods and instructions for each method, see Available options to connect with AWS.
    To gain permission to collect all AWS Bedrock attributes, add the following permissions to your AWS IAM policy JSON snippet:
    "bedrock:ListTagsForResource",
"bedrock:ListFoundationModels",
"bedrock:GetFoundationModel",
"bedrock:ListInferenceProfiles"
  2. Run your app.py file.
    You can call your AWS Bedrock models within your application either on local or on cloud. The following command calls a local Python application file with model invocations. 
    python app.py
  3. Use the AWS Bedrock navigator to monitor your AWS Bedrock applications. For more information on navigators, see Use navigators.
    1. From the Splunk Observability Cloud main menu, select Infrastructure.
    2. Under AI/ML, select AI Frameworks.
    3. Select the AWS Bedrock summary card.

Metrics

Learn about the available metrics for AWS Bedrock.

The following metrics are available for AWS Bedrock applications. For more information on these metrics, see Monitoring the performance of Amazon Bedrock in the AWS documentation and View organization metrics for Splunk Observability Cloud.
Metric nameTypeUnitDescription
InputTokenCountcountercountInput tokens consumed.
OutputTokenCountcountercountOutput tokens consumed.
InvocationscounterSampleCountThe number of successful requests made to AWS Bedrock API operations.
InvocationLatencyhistogrammsThe time taken for an invocation, specifically the latency of the requests to the AWS Bedrock models, typically in milliseconds.
sf.org.num.awsServiceAuthErrorCount gaugecount

Total number of authentication errors thrown by AWS services.

  • Dimension(s): integrationId, orgId, namespace (AWS Cloudwatch namespace), clientInterface (AWS SDK Interface), method (AWS SDK method)

  • Data resolution: 1 second

sf.org.num.awsServiceCallThrottlesgaugecount

Number of calls made to the Amazon API that are being throttled by AWS because you have exceeded your AWS API call limits.

  • Dimension(s): orgId, namespace (AWS service, such as AWS/Cloudwatch), method (the API being called, such as getMetricStatistics)

  • Data resolution: 5 seconds

sf.org.num.awsServiceCallThrottledRetryFailuresgaugecountNumber of AWS service call throttle retry failures.
sf.org.num.awsServiceCallCountExceptionsgaugecount

Number of calls made to the Amazon API that threw exceptions.

  • Dimension(s): orgId, namespace (AWS service, such as AWS/Cloudwatch), method (the API being called, such as getMetricStatistics), location (where the exception occurred, either client or server)

  • Data resolution: 5 seconds

sf.org.num.awsServiceCallCountgaugecount

Total number of calls made to the Amazon API.

  • Dimension(s): orgId, namespace (AWS service, such as AWS/Cloudwatch), method (the API being called, such as getMetricStatistics)

  • Data resolution: 5 seconds

Attributes

Learn about the available resource attributes for AWS Bedrock applications.

The following resource attribute is available for AWS Bedrock applications.

OpenTelemetry nameDisplay nameDescription
ididThe AWS unique identifier.

Troubleshoot

Learn how to get help if you can't see your data in Splunk Observability Cloud.

If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways:

  • Prospective customers and free trial users can ask a question and get answers through community support in the Splunk Community.