Monitor AWS Bedrock APIs

Splunk AppDynamics Python Agent helps you monitor the AWS Bedrock usage and related metrics. It monitors the converse and converse_stream requests created using the Boto3 library.

The following metrics are reported at the application, tier, and node levels:

  • Input Tokens
  • Completion Tokens
  • Tokens
  • Latency
  • Calls per minute

Prerequisites

Python agent supports the Boto3 for AWS Bedrock in Splunk AppDynamics. For more information, see Boto3 documentation.

Ensure that you meet the following prerequisites to monitor the OpenAI API exit calls with AppDynamics:

Create Custom Dashboard

  1. Download Custom_dashboard_script.zip. It contains:
    DirectoryDescription
    templatesContains json templates to create the dashboard files.
    dashboardDashboard json files are created here. It contains the sample json files.
    main.pyPython script to create the dashboards.
    readme.mdDescription of the solution.
  2. Run the following command:
    cd Custom_dashboard_script
python3 main.py

    The preceding command prompts you to specify the dashboard type (application or tier) that you want to create.

  3. Based on your selected dashboard type, you will be prompted to specify the following dashboard details.
    Application Level Dashboard Detals
    VariableDescriptionDefaultPossible ValuesRequired
    Dashboard NameDashboard Title that is displayed in Dashboard and Reports.bedrock_app_dashboardbedrock_app_dashboardOptional
    Application NameName of the Application with bedrock backend calls.bedrock_appNoneYes
    Model Level Dashboard Details
    VariableDescriptionDefaultPossible ValuesRequired
    Model NameName of the Model which you want to monitor.amazon-titan-dashbaordNoneYes
    Dashboard NameDashboard Title that is displayed in Dashboard and Reports.bedrock_app_dashboardbedrock_app_dashboardOptional
    Application NameName of the Application with bedrock backend calls.bedrock_appNoneYes
    Note: If Model Name has special characters, you must replace them with hyphens ( - ). For example, if the model name is "amazon-titan-lite:1", change the name to "amazon-titan-lite-1".
  4. Import the dashboard in the Controller UI. See Create Custom Dashboard
    Tip: Navigate to Tiers & Nodes > My Dashboards and create a dashboard for the selected application by copying and pasting widgets from the existing dashboard for easier navigation.