Configure the Prometheus receiver to collect Kong AI Gateway Proxy metrics

Send Kong AI Gateway Proxy metrics to Splunk Observability Cloud.

You can monitor the performance of your Kong AI Gateway Proxy by configuring the Splunk Distribution of the OpenTelemetry Collector to send Kong AI Gateway Proxy metrics to Splunk Observability Cloud.

This solution uses the Prometheus receiver to collect metrics from Kong AI Gateway MCP Proxy, which exposes the Prometheus-compatible /metrics endpoint.

To configure the Prometheus receiver to collect Kong AI Gateway Proxy metrics, you must meet the following requirements.

  • You're using a supported Kong AI Gateway version for the metrics you want to monitor.

    • LLM metrics are generally available for Kong AI Gateway, but specific latency metrics require Kong AI Gateway 3.8 and higher.

    • MCP metrics require Kong AI Gateway 3.12 and higher.

  • You've enabled AI metrics for Kong AI Gateway. This step is required to collect AI metrics such as requests, costs, tokens, and MCP traffic. To enable them:

  • Your metrics are exposed in Prometheus format at the /metrics endpoint. This endpoint must be accessible with the Status API (preferred method; requires status_listen to be enabled) or the Admin API. For instructions, see Accessing the metrics in the Kong documentation.

  1. Deploy the Splunk Distribution of the OpenTelemetry Collector to your host or container platform:
  2. To manually activate the Prometheus receiver for Kong AI Gateway Proxy, make the following changes to your Collector values.yaml configuration file.
    1. Add prometheus/kong-ai-gateway to the receivers section. For example:
      YAML 
      agent:
  config:
    receivers:
      prometheus/kong-ai-gateway:
        config:
          scrape_configs:
          - job_name: 'kong-ai-gateway-proxy'
            scrape_interval: 15s 
            metrics_path: /metrics
            static_configs:
            # Note that the URL to API is specific to your installation
            - targets: ['http://{host}:{port}']
    2. Add prometheus/kong-ai-gateway to the metrics pipeline of the service section. For example:
      YAML 
      service:
  pipelines:
    metrics:
      receivers:
        - prometheus/kong-ai-gateway
  3. Restart the Splunk Distribution of the OpenTelemetry Collector.

Configuration settings

To view the configuration options for the Prometheus receiver, see Settings.

Metrics

The following metrics are available for Kong AI Gateway Proxy. For more information, see Monitor AI LLM metrics in the Kong documentation.

LLM metrics

Metric name Type Description
kong_ai_llm_cost_total counter AI requests cost per ai_provider/cache in Kong.
kong_ai_llm_provider_latency_ms histogram LLM response latency for each AI plugin per ai_provider in Kong.
kong_ai_llm_requests_total counter AI requests total per ai_provider in Kong.
kong_ai_llm_tokens_total counter AI requests cost per ai_provider/cache in Kong.

MCP metrics

Attribute name Type Description
kong_ai_mcp_error_total counter Total MCP server errors by type.
kong_ai_mcp_latency_ms histogram MCP server latencies in milliseconds.
kong_ai_mcp_response_body_size_bytes histogram MCP server response body sizes in bytes.

Attributes

The following resource attributes are available for Kong AI Gateway Proxy.

LLM attributes

Attribute name Description
ai_provider The upstream LLM/AI provider routing traffic through Kong.
ai_model The specific model served by the AI provider.
cache_status Semantic cache hit/miss status. Empty if no cache is configured.
vector_db Vector database used for semantic caching. Empty if not used.
embeddings_provider Provider used to generate embeddings for cache lookups.
embeddings_model Model used to generate embeddings for cache lookups.
workspace Kong workspace isolating the configuration.
token_type The type of token count being tracked. Only available for the kong_ai_llm_tokens_total metric.

MCP attributes

Attribute name Description
service Name of the Kong service routing to the MCP server.
route Kong route matched for the MCP request.
method Invoked MCP JSON-RPC method.
workspace Kong workspace isolating the configuration.
tool_name Name of the MCP tool being called. Empty on errors before dispatch.
type Category/type of MCP error encountered. Only available for the kong_ai_mcp_error_total metric.