Configure the Splunk Distribution of OTel JS for Splunk Observability Cloud
Configure the Splunk Distribution of OpenTelemetry JS to suit your instrumentation needs, like correlating traces with logs, activating exporters, and more.
You can configure the Splunk Distribution of OpenTelemetry JS to suit your instrumentation needs. In most cases, modifying the basic configuration is enough to get started.
The following sections describe all available settings for configuring OpenTelemetry for Node.js, including options for activating new features that are unique to the Splunk Distribution of OpenTelemetry JS.
Configuration methods
To configure the Splunk Distribution of OpenTelemetry JS, you can use a combination of environment variables and arguments passed to the start() function:
Environment variables
For example:
export OTEL_SERVICE_NAME='test-service'Arguments passed to the
start()functionFor example:
start({ serviceName: 'my-node-service', });
Configuration for each of the supported data type, such as metrics or tracing, is set using additional properties on the configuration object:
start({
// general options like `serviceName` and `endpoint`
metrics: {
// configuration passed to metrics signal
},
profiling: {
// configuration passed to profiling signal
},
tracing: {
// configuration passed to tracing signal
},
});You can also activate the collection of a specific data type by passing a boolean value instead of an object. For example:
start({
// general options like `serviceName` and `endpoint`
metrics: true, // turn metrics on with default options
profiling: true, // turn profiling on with default options
});Resource configuration
Starting with version 4.0 of the Splunk Distribution of OpenTelemetry JS, manual resource creation using new Resource({}) is no longer supported.
To define custom resource attributes, use the resourceFromAttributes() helper from @opentelemetry/resources and merge it with the detected resource:
import { start } from '@splunk/otel';
import { resourceFromAttributes } from '@opentelemetry/resources';
start({
resource: (detectedResource) => {
return detectedResource.merge(resourceFromAttributes({
'my.attribute': 'foo',
}));
},
});This approach ensures compatibility with the latest OpenTelemetry SDK and allows you to define custom resource attributes programmatically.
General settings
The following settings are specific to the Splunk Distribution of OpenTelemetry JS:
| Description | Environment variable | Default | Type | Argument to start() |
|---|---|---|---|---|
| Log level for the OTel diagnostic logger. | OTEL_LOG_LEVEL | none | string | logLevel |
| Name of the service or application. | OTEL_SERVICE_NAME | unnamed-node-service | string | serviceName |
| Splunk authentication token. | SPLUNK_ACCESS_TOKEN | string | accessToken | |
| The name of your organization’s realm. | SPLUNK_REALM | string | realm | |
| Adds server trace info to HTTP response headers. | SPLUNK_TRACE_RESPONSE_HEADER_ENABLED | true | boolean | tracing.serverTimingEnabled |
Instrumentations configuration
The following settings control which instrumentations are activated:
| Description | Environment variable | Default | Type | Argument to start() |
|---|---|---|---|---|
| The delay in milliseconds between 2 consecutive batch span processor exports. | OTEL_BSP_SCHEDULE_DELAY | 5000 | number | |
| Activates all embedded instrumentations. | OTEL_INSTRUMENTATION_COMMON_DEFAULT_ENABLED | true | boolean | |
| Activates a specific instrumentation. | OTEL_INSTRUMENTATION_<NAME>_ENABLED | true | boolean | |
| Emit metrics from instrumentation (for example, http.server.duration). | SPLUNK_INSTRUMENTATION_METRICS_ENABLED | false | boolean | |
| Whether to include the full Redis query in db.statement span attributes. | SPLUNK_REDIS_INCLUDE_COMMAND_ARGS | false | boolean | |
| Enables GraphQL resolver spans. | SPLUNK_GRAPHQL_RESOLVE_SPANS_ENABLED | false | boolean | |
| Enables tracing. | SPLUNK_TRACING_ENABLED | true | boolean |
For example, to turn off all default instrumentations and only turn on the bunyan instrumentation, set the following environment variables:
export OTEL_INSTRUMENTATION_COMMON_DEFAULT_ENABLED=false
export OTEL_INSTRUMENTATION_BUNYAN_ENABLED=trueThe previous settings only apply to instrumentations loaded by the Splunk Distribution of OpenTelemetry JS by default. When using the programmatic API to supply a list of user-specified instrumentations, they have no effect.
Trace configuration
The following settings control tracing limits and attributes:
Environment variable | Argument to start() | Description |
|---|---|---|
OTEL_TRACE_ENABLED | Not applicable | Activates tracer creation and autoinstrumentation. Default value is |
OTEL_SERVICE_NAME |
| Name of the service or application you’re instrumenting. Takes precedence over the service name defined in the |
OTEL_RESOURCE_ATTRIBUTES | Not applicable | Comma-separated list of resource attributes added to every reported span. For example, |
OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | Not applicable | Maximum number of attributes per span. Default value is unlimited. |
OTEL_SPAN_EVENT_COUNT_LIMIT | Not applicable | Maximum number of events per span. Default value is unlimited. |
OTEL_SPAN_LINK_COUNT_LIMIT | Not applicable | Maximum number of links per span. Default value is |
OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT | Not applicable | Maximum length of strings for attribute values. Values larger than the limit are truncated. Default value is |
Samplers configuration
The following settings control trace sampling:
Environment variable | Description |
|---|---|
OTEL_TRACES_SAMPLER | Sampler to use. The default value is |
OTEL_TRACES_SAMPLER_ARG | Semicolon-separated list of rules for the |
Exporters configuration
The following settings control trace exporters and their endpoints:
| Description | Environment variable | Default | Type | Argument to start() |
|---|---|---|---|---|
| Path to a certificate to use when verifying a server's TLS credentials. | OTEL_EXPORTER_OTLP_CERTIFICATE | string | ||
| Path to a certificate to use when verifying a client's TLS credentials. | OTEL_EXPORTER_OTLP_CLIENT_CERTIFICATE | string | ||
| Path to client's private key to use in mTLS communication in PEM format. | OTEL_EXPORTER_OTLP_CLIENT_KEY | string | ||
| The OTLP endpoint to export to. | OTEL_EXPORTER_OTLP_ENDPOINT | http://localhost:4318 | string | endpoint |
| Chooses the trace exporter protocol. Allowed values are grpc and http/protobuf. | OTEL_EXPORTER_OTLP_TRACES_PROTOCOL | http/protobuf | string | |
| Chooses the metric exporter protocol. Allowed values are grpc and http/protobuf. | OTEL_EXPORTER_OTLP_METRICS_PROTOCOL | http/protobuf | string | metrics.metricReaderFactory |
| The protocol to use for OTLP exports. | OTEL_EXPORTER_OTLP_PROTOCOL | http/protobuf | string | |
| The traces OTLP endpoint to export to. | OTEL_EXPORTER_OTLP_TRACES_ENDPOINT | http://localhost:4318 | string | |
| Comma-separated list of metrics exporters to use. | OTEL_METRICS_EXPORTER | otlp | string | metrics.metricReaderFactory |
| Comma-separated list of trace exporters to use. | OTEL_TRACES_EXPORTER | otlp | string | tracing.spanExporterFactory |
Jaeger exporter
To use the Jaeger exporter, add the @opentelemetry/exporter-jaeger package as in the following example:
const { start } = require('@splunk/otel');
const { JaegerExporter } = require('@opentelemetry/exporter-jaeger');
start({
serviceName: 'my-node-service',
tracing: {
spanExporterFactory: (options) => {
return new JaegerExporter({
serviceName: options.serviceName,
// Additional config
})
}
},
});Propagators configuration
The following settings control trace propagation:
| Description | Environment variable | Default | Type | Argument to start() |
|---|---|---|---|---|
| Comma-separated list of propagators you want to use. | OTEL_PROPAGATORS | tracecontext,baggage | string | tracing.propagators |
For backward compatibility with the SignalFx Tracing Library for Node.js, use the b3multi trace propagator:
- Linux
export OTEL_PROPAGATORS=b3multi- Windows PowerShell
$env:OTEL_PROPAGATORS=b3multi
Node.js settings for AlwaysOn Profiling
The following settings control the AlwaysOn Profiling feature for the Node.js agent:
| Description | Environment variable | Default | Type | Argument to start() |
|---|---|---|---|---|
| Activates AlwaysOn CPU profiling. | SPLUNK_PROFILER_ENABLED | false | boolean | profilingEnabled |
| The collector endpoint for profiler logs. | SPLUNK_PROFILER_LOGS_ENDPOINT | http://localhost:4318 | string | profiling.endpoint |
| Activates memory profiling for AlwaysOn Profiling. | SPLUNK_PROFILER_MEMORY_ENABLED | false | string | profiling.memoryProfilingEnabled |
| Frequency with which call stacks are sampled, in milliseconds. | SPLUNK_PROFILER_CALL_STACK_INTERVAL | 1000 | number | profiling.callstackInterval |
To configure AlwaysOn Profiling programmatically, pass the arguments to the start function, as in the following example:
start({
serviceName: '<service-name>',
profiling: true,
tracing: {
// configuration passed to tracing signal
},
});Metrics configuration
The following settings activate runtime metrics collection:
| Description | Environment variable | Default | Type | Argument to start() |
|---|---|---|---|---|
| The metrics endpoint. Takes precedence over the value set in OTEL_EXPORTER_OTLP_ENDPOINT. | OTEL_EXPORTER_OTLP_METRICS_ENDPOINT | https://ingest.<realm>.signalfx.com/v2/datapoint/otlp | number | metrics.endpoint |
| The interval, in milliseconds, of metrics collection and exporting. | OTEL_METRIC_EXPORT_INTERVAL | 30000 | number | metrics.exportIntervalMillis |
| Activates metrics collection. | SPLUNK_METRICS_ENABLED | false | boolean | Activated by calling start() |
| The interval, in milliseconds, during which GC and event loop statistics are collected. | SPLUNK_RUNTIME_METRICS_COLLECTION_INTERVAL | 5000 | number | metrics.runtimeMetricsCollectionIntervalMillis |
| Activates the collection and export of runtime metrics. | SPLUNK_RUNTIME_METRICS_ENABLED | true | boolean | metrics.runtimeMetricsEnabled |
| Activates the collection and export of internal debug metrics for troubleshooting. | SPLUNK_DEBUG_METRICS_ENABLED | true | boolean | metrics.debugMetricsEnabled |
start() function.Configuring an existing metrics client to send custom metrics
You can use an existing SignalFx client for sending custom metrics instead of creating and configuring a new one.
To configure an existing client, pass the following data to the start() function:
signalfx: A JavaScript object with optionalclientanddimensionsfields. Thedimensionsobject adds a predefined dimension for each data point. The format fordimensionsis{key: value, ...}.
The following is a list of dimensions added by default:
service: SeeserviceNamein Trace configuration.metric_source:splunk-otel-jsnode_version:process.versions.node, for example16.10.0
Server trace information
To connect Real User Monitoring (RUM) requests from mobile and web applications with server trace data, activate Splunk trace response headers by setting the following environment variable:
- Linux
export SPLUNK_TRACE_RESPONSE_HEADER_ENABLED=true- Windows PowerShell
$env:SPLUNK_TRACE_RESPONSE_HEADER_ENABLED=true
When you set this environment variable, your application instrumentation adds the following response headers to HTTP responses.
Access-Control-Expose-Headers: Server-Timing
Server-Timing: traceparent;desc="00-<serverTraceId>-<serverSpanId>-01"The Server-Timing header contains the traceId and spanId in traceparent format. For more information, see the Server-Timing and traceparent documentation on the W3C website.