PostgreSQL receiver

The PostgreSQL receiver is a component of the OpenTelemetry Collector. It connects to a PostgreSQL instance and supports metrics and logs pipelines.

PostgreSQL support starts from these minimum collector versions:

• Splunk Distribution of the OpenTelemetry Collector (splunk-otel-collector) v0.147.0 or later
• Community (OSS) version of the OpenTelemetry Collector (opentelemetry-collector-contrib) v0.147.0 or later

Supported versions and platforms

Splunk Database Monitoring supports these PostgreSQL versions and platforms:

Prerequisites

1. Create a monitoring user for the receiver. For Azure Database for PostgreSQL Flexible Server, Amazon RDS for PostgreSQL, Amazon Aurora PostgreSQL, self-hosted PostgreSQL, and EDB Postgres Advanced Server, use:

   SQLCopy

   CREATE USER "otel-user" WITH PASSWORD 'otel-user-password'; GRANT pg_monitor TO "otel-user"; GRANT SELECT ON pg_stat_database TO "otel-user";

   CREATE USER "otel-user" WITH PASSWORD 'otel-user-password';
   GRANT pg_monitor TO "otel-user";
   GRANT SELECT ON pg_stat_database TO "otel-user";

   To use AWS IAM database authentication for Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL, create the monitoring user without a password and grant it the rds_iam role:

   SQLCopy

   CREATE USER "otel-user" WITH LOGIN; GRANT rds_iam TO "otel-user"; GRANT pg_monitor TO "otel-user"; GRANT SELECT ON pg_stat_database TO "otel-user";

   CREATE USER "otel-user" WITH LOGIN;
   GRANT rds_iam TO "otel-user";
   GRANT pg_monitor TO "otel-user";
   GRANT SELECT ON pg_stat_database TO "otel-user";

   Note: For PostgreSQL, after a user has the rds_iam role, IAM authentication takes precedence over password authentication for that user. Create a separate user if you also need password-based access.

2. Configure the database:

   AWS PostgreSQL

   Tip: By default, AWS PostgreSQL parameter groups already include pg_stat_statements in shared_preload_libraries. If the parameter group has been modified, perform a check to ensure pg_stat_statements is included in shared_preload_libraries.

   • For Amazon Aurora PostgreSQL, configure the receiver with the instance endpoint, not the cluster endpoint.

   • For Amazon AWS RDS PostgreSQL, configure the receiver with the primary instance endpoint

   • To collect query plans for application queries, grant the monitoring user SELECT on the relevant application schemas and tables.

   • To create the extension in each target database, connect to each database the receiver will scrape and run this command:

     SQLCopy

     CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

     CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

   Azure PostgreSQL

   1. Update server parameters in Azure Portal:

      1. In Azure Portal, navigate to PosgreSQM Flexible Server > Settings > Server parameters and set the following parameters:

         Note: If a parameter already has values, use a comma-separated list and add pg_stat_statements instead of removing the existing values:

         • azure.extensions = pg_stat_statements

         • shared_preload_libraries = pg_stat_statements

         • pg_stat_statements.track = all

         • pg_stat_statements.max = 10000

         • pg_stat_statements.track_utility = on

      2. Select Save.

      3. Restart the server.

   2. To create the extension in each target database, connect to each database the receiver will scrape and run this command:

      CODECopy

      CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

      CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

      Sample connection:

      CODECopy

      psql "host=server.postgres.database.azure.com \ port=5432 \ dbname=postgres \ user=otel-user \ sslmode=require"

      psql "host=server.postgres.database.azure.com \
            port=5432 \
            dbname=postgres \
            user=otel-user \
            sslmode=require"

   EDB Postgres Advanced Server

   To use the PostgreSQL receiver with EDB Postgres Advanced Server, perform the following steps.

   • Add pg_stat_statements to shared_preload_libraries in postgresql.conf. Preserve existing libraries in the list. For example:

     SQLCopy

     shared_preload_libraries = '$libdir/dbms_pipe,$libdir/edb_gen,$libdir/dbms_aq,pg_stat_statements'

     shared_preload_libraries = '$libdir/dbms_pipe,$libdir/edb_gen,$libdir/dbms_aq,pg_stat_statements'

     Restart EDB Postgres Advanced Server after changing postgresql.conf.

   • To create the extension in each target database, connect to each database the receiver will scrape and run this command:

     SQLCopy

     CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

     CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

     To collect query plans for application queries, grant the monitoring user SELECT on the relevant application schemas and tables.

   • If EDB Postgres Advanced Server allows only local connections, update pg_hba.conf to allow connections from the collector source network. For TLS-enabled connections, use the appropriate hostssl rule. For non-TLS connections, use a host rule.

   Self-hosted PostgreSQL

   To use the PostgreSQL receiver with self-hosted PostgreSQL, perform the following steps.

   1. Enable pg_stat_statements

      top_query requires pg_stat_statements to be loaded at server startup. If your environment supports server configuration changes, do this:

      1. Add this line to postgresql.conf:

         SQLCopy

         shared_preload_libraries = 'pg_stat_statements'

         shared_preload_libraries = 'pg_stat_statements'

      2. Restart PostgreSQL.

      3. If your environment supports ALTER SYSTEM, you can also use:

         SQLCopy

         ALTER SYSTEM SET shared_preload_libraries = 'pg_stat_statements';

         ALTER SYSTEM SET shared_preload_libraries = 'pg_stat_statements';

      4. Restart PostgreSQL.

   2. To create the extension in each target database, connect to each database the receiver will scrape and run this command:

      SQLCopy

      CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

      CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

Configure the receiver

Modify your collector configuration file as follows. All examples are for the Splunk Distribution of the OpenTelemetry Collector.

1. In the receivers: section, add postgresql:

   YAMLCopy

   postgresql: collection_interval: 10s databases: - postgres endpoint: your-service-endpoint:5432 events: db.server.query_sample: enabled: true db.server.top_query: enabled: true username: otel-user password: otel-user-password tls: insecure: true

   postgresql:
     collection_interval: 10s
     databases:
       - postgres
     endpoint: your-service-endpoint:5432
     events:
       db.server.query_sample:
         enabled: true
       db.server.top_query:
         enabled: true
     username: otel-user
     password: otel-user-password
     tls:
       insecure: true

   Note: On Amazon RDS for PostgreSQL and Amazon Aurora PostgreSQL, you can replace password with AWS IAM database authentication. IAM authentication doesn't store a database password in the collector configuration, and requires tls.insecure: false. See Authenticate to Amazon RDS or Aurora with AWS IAM.
   Important:

   If you're using the Splunk Distribution of OpenTelemetry Collector, leave the following receiver settings at their default values:

   • collection_interval (Default: 10s)

   • query_sample_collection.max_rows_per_query (Default: 100)

   • top_query_collection.collection_interval (Default: 60s)

   • top_query_collection.max_query_sample_count (Default: 1000)

   These values support Database Monitoring without affecting the performance of the database or the collector. If you increase these values you might adversely affect the performance of your database or collector, and this could result in ingest throttling.

2. In the exporters: section, add otlp_http/dbmon:

   YAMLCopy

   otlp_http/dbmon: headers: X-SF-Token: your-splunk-access-token X-splunk-instrumentation-library: dbmon logs_endpoint: https://ingest.your-splunk-realm.observability.splunkcloud.com/v3/event sending_queue: batch: flush_timeout: 15s max_size: 10485760 sizer: bytes

   otlp_http/dbmon:
     headers:
       X-SF-Token: your-splunk-access-token
       X-splunk-instrumentation-library: dbmon
     logs_endpoint: https://ingest.your-splunk-realm.observability.splunkcloud.com/v3/event
     sending_queue:
       batch:
         flush_timeout: 15s
         max_size: 10485760
         sizer: bytes

3. In the service.pipelines: section, create a metrics pipeline named metrics/dbmon and a logs pipeline named logs/dbmon:

   YAMLCopy

   metrics/dbmon: receivers: - postgresql processors: - memory_limiter - batch exporters: - signalfx logs/dbmon: receivers: - postgresql processors: - memory_limiter - batch exporters: - otlp_http/dbmon

   metrics/dbmon:
     receivers:
       - postgresql
     processors:
       - memory_limiter
       - batch
     exporters:
       - signalfx
   logs/dbmon:
     receivers:
       - postgresql
     processors:
       - memory_limiter
       - batch
     exporters:
       - otlp_http/dbmon

   Important: Use an identical list of processors for the metrics and logs/dbmon pipelines, and include these processors in the same order.

4. Restart the collector to apply your configuration changes.

   The restart command varies depending on what platform you deployed the collector on and what tool you used to deploy it. Here are general examples of the restart command:

   Linux

   Linux with installer script:

   BASHCopy

   sudo systemctl restart splunk-otel-collector

   sudo systemctl restart splunk-otel-collector

   Windows

   Windows with installer script:

   BASHCopy

   stop-service splunk-otel-collector start-service splunk-otel-collector

   stop-service splunk-otel-collector
   start-service splunk-otel-collector

   Kubernetes

   Kubernetes with Helm:

   BASHCopy

   helm upgrade your-splunk-otel-collector splunk-otel-collector-chart/splunk-otel-collector -f your-override-values.yaml

   helm upgrade your-splunk-otel-collector splunk-otel-collector-chart/splunk-otel-collector -f your-override-values.yaml

   where splunk-otel-collector-chart is the name you gave to the Helm chart in the helm repo add command.

Your database instance should now be visible on APM > Database monitoring as well as on Infrastructure > Infrastructure monitoring if you have a Database Monitoring license. For troubleshooting, see Troubleshoot data collection .

Advanced configurations

Collect data from multiple database instances

Omit the database name parameter, receivers.postgresql.database. If omitted, the receiver collects metrics from all instances.

Identify database instances uniquely

Use service.instance.id, service.name, and service.namespace resource attributes to distinguish PostgreSQL instances when the generated identifier doesn't identify each instance uniquely in your deployment.

To identify database instances by using a combination of service.name, service.namespace, and override_value, use Splunk Distribution of the OpenTelemetry Collector (splunk-otel-collector) version 0.155.0 or higher or Community (OSS) version of the OpenTelemetry Collector (opentelemetry-collector-contrib) version 0.155.0 or higher.

The receiver sets service.instance.id from the configured endpoint. In Kubernetes and other environments, the same host and port can appear in multiple clusters. Set service.namespace to the cluster or environment and service.name to the logical database service.

If you collect metrics and logs, add the resource attributes under both metrics.resource_attributes and logs.resource_attributes. For example:

YAMLCopy

postgresql: endpoint: your-service-endpoint:5432 username: otel-user password: otel-user-password transport: tcp metrics: resource_attributes: service.instance.id: enabled: true override_value: your-service-endpoint:5432 service.name: enabled: true override_value: payments-db service.namespace: enabled: true override_value: prod-us-east-k8s logs: resource_attributes: service.instance.id: enabled: true override_value: your-service-endpoint:5432 service.name: enabled: true override_value: payments-db service.namespace: enabled: true override_value: prod-us-east-k8s

postgresql:
  endpoint: your-service-endpoint:5432
  username: otel-user
  password: otel-user-password
  transport: tcp
  metrics:
    resource_attributes:
      service.instance.id:
        enabled: true
        override_value: your-service-endpoint:5432
      service.name:
        enabled: true
        override_value: payments-db
      service.namespace:
        enabled: true
        override_value: prod-us-east-k8s
  logs:
    resource_attributes:
      service.instance.id:
        enabled: true
        override_value: your-service-endpoint:5432
      service.name:
        enabled: true
        override_value: payments-db
      service.namespace:
        enabled: true
        override_value: prod-us-east-k8s

Note: Set override_value only when enabled: true. If enabled is false, override_value has no effect. When enabled without override_value, service.name defaults to unknown_service:postgresql and service.namespace defaults to an empty string.

Authenticate to Amazon RDS or Aurora with AWS IAM

By default, the receiver authenticates with the password setting. On Amazon RDS for PostgreSQL and Amazon Aurora PostgreSQL, you can use AWS IAM database authentication instead. With IAM authentication, the collector creates a short-lived authentication token for each database connection instead of storing a database password in the collector configuration.

This option requires Splunk Distribution of the OpenTelemetry Collector (splunk-otel-collector) version 0.159.0 or higher, or Community (OSS) version of the OpenTelemetry Collector (opentelemetry-collector-contrib) version 0.159.0 or higher.

Before you configure the receiver, complete the AWS and database prerequisites. For AWS setup instructions, see IAM database authentication for MariaDB, MySQL, and PostgreSQL in the Amazon RDS User Guide. At minimum, turn on IAM database authentication, create a database user with the rds_iam role, and attach an IAM policy that grants rds-db:connect to the IAM role that runs the collector.

For the IAM policy resource ARN, use the DbiResourceId of the DB instance for Amazon RDS, or the DbClusterResourceId of the cluster for Amazon Aurora. The final path segment is the database user name and is case-sensitive.

Configure IAM authentication in the collector:

1. In the extensions: section, add aws_iam_db_auth and set region to the AWS Region of your database. The region setting is required.

2. In the receivers: section, set db_auth to aws_iam_db_auth and remove the password setting. Keep username. The receiver passes the user name to the extension so the extension can create the token.

3. Set tls.insecure to false. IAM database authentication requires an encrypted connection.

4. In the service.extensions: section, add aws_iam_db_auth to the existing list of extensions.

For example:

YAMLCopy

extensions: aws_iam_db_auth: region: us-east-1 receivers: postgresql: collection_interval: 10s databases: - postgres endpoint: your-service-endpoint:5432 events: db.server.query_sample: enabled: true db.server.top_query: enabled: true username: otel-user db_auth: aws_iam_db_auth tls: insecure: false insecure_skip_verify: false ca_file: /etc/ssl/certs/rds-global-bundle.pem service: extensions: - aws_iam_db_auth pipelines: metrics/dbmon: receivers: - postgresql processors: - memory_limiter - batch exporters: - signalfx

extensions:
  aws_iam_db_auth:
    region: us-east-1

receivers:
  postgresql:
    collection_interval: 10s
    databases:
      - postgres
    endpoint: your-service-endpoint:5432
    events:
      db.server.query_sample:
        enabled: true
      db.server.top_query:
        enabled: true
    username: otel-user
    db_auth: aws_iam_db_auth
    tls:
      insecure: false
      insecure_skip_verify: false
      ca_file: /etc/ssl/certs/rds-global-bundle.pem

service:
  extensions:
    - aws_iam_db_auth
  pipelines:
    metrics/dbmon:
      receivers:
        - postgresql
      processors:
        - memory_limiter
        - batch
      exporters:
        - signalfx

Important: db_auth and password are mutually exclusive. If you set both, the collector fails to start with invalid config: set either 'password' or 'db_auth', not both. Remove password when you use db_auth.

Note: The service.extensions list in the example is abbreviated. Add aws_iam_db_auth to the extensions that your configuration already declares. An extension that is configured but missing from service.extensions doesn't start, and the receiver fails with db_auth: requested credential provider is not present.

If you compare this configuration to AWS PostgreSQL documentation that uses sslmode, use the receiver tls settings to configure the equivalent behavior.

tls setting	Resulting sslmode	IAM authentication support
insecure: true	disable	Not supported
insecure: false and insecure_skip_verify: true	require	Supported
insecure: false and insecure_skip_verify: false	verify-full	Supported and recommended

AWS recommends verify-full with the Amazon RDS certificate bundle. Set insecure_skip_verify: false and point ca_file at the certificate bundle. To download the bundle, see Using SSL/TLS to encrypt a connection to a DB instance or cluster in the Amazon RDS User Guide.

The sample in Configure the receiver uses tls.insecure: true. That setting disables TLS and isn't compatible with IAM authentication.

To monitor databases in more than one AWS Region, declare one named instance of the extension for each region and point each receiver at the extension instance it needs:

YAMLCopy

extensions: aws_iam_db_auth: region: us-east-1 aws_iam_db_auth/west: region: us-west-2 receivers: postgresql/east: endpoint: your-service-endpoint:5432 username: otel-user db_auth: aws_iam_db_auth tls: insecure: false postgresql/west: endpoint: your-service-endpoint:5432 username: otel-user db_auth: aws_iam_db_auth/west tls: insecure: false service: extensions: - aws_iam_db_auth - aws_iam_db_auth/west

extensions:
  aws_iam_db_auth:
    region: us-east-1
  aws_iam_db_auth/west:
    region: us-west-2

receivers:
  postgresql/east:
    endpoint: your-service-endpoint:5432
    username: otel-user
    db_auth: aws_iam_db_auth
    tls:
      insecure: false
  postgresql/west:
    endpoint: your-service-endpoint:5432
    username: otel-user
    db_auth: aws_iam_db_auth/west
    tls:
      insecure: false

service:
  extensions:
    - aws_iam_db_auth
    - aws_iam_db_auth/west

Databases in the same AWS Region can share one extension instance. To add another database in the same Region, add another receiver with the same db_auth value.

Tip:

Connection pooling and authentication token lifecycle (receiver.postgresql.connectionPool)

AWS IAM database authentication tokens have a 15-minute validity window. Note the following behavior when using connection pooling with the PostgreSQL receiver:

• Existing connections: Open connections in the connection pool remain active and functional for continuous metric collection, even after the authentication token used to open them expires. Active connections don't require periodic token refreshes.
• New connections: If a connection closes or the collector opens additional connections, the collector automatically requests a fresh IAM authentication token to establish the new session.

Use the following table to troubleshoot AWS IAM authentication errors:

Message in the collector log	Cause and resolution
invalid config: set either 'password' or 'db_auth', not both	Both credential settings are present. Remove password.
db_auth: requested credential provider is not present: "aws_iam_db_auth"	The extension is configured but not listed in service.extensions. Add it.
db_auth: requested extension is not a credential provider	db_auth names an extension that doesn't provide database credentials. Set it to an aws_iam_db_auth instance.
aws_iam_db_auth: region must be set on the extension	Add the required region setting to the extension.
aws_iam_db_auth: mint RDS token for ...	The collector can't resolve AWS credentials. Confirm that the instance profile, task role, or IAM role for service accounts (IRSA) is attached and reachable.
pq: PAM authentication failed for user "otel-user"	The AWS setup is incomplete. Confirm that IAM database authentication is turned on for the instance, the database user has the rds_iam role, the IAM policy allows rds-db:connect for that user name, and the extension region matches the database Region.
pq: SSL is not enabled on the server, or a pg_hba.conf error that mentions SSL is off	TLS is disabled. Set tls.insecure to false. IAM database authentication requires an encrypted connection.

Enable optional metrics

Set metrics.metric-name.enabled to true. For example:

YAMLCopy

postgresql: metrics: postgresql.tup_fetched: enabled: true

postgresql:
  metrics:
    postgresql.tup_fetched:
      enabled: true

Set up APM correlation

See Correlate database queries with Splunk APM traces.

Settings reference

Metrics reference