Configure the Splunk Distribution of OpenTelemetry .NET
Configure the Splunk Distribution of OpenTelemetry .NET to suit your instrumentation needs, such as correlating traces with logs and activating custom sampling.
You can configure the Splunk Distribution of OpenTelemetry .NET to suit your instrumentation needs. In most cases, modifying the basic configuration is enough to get started. More advanced settings are also available.
Configuration methods
You can change the settings of the Splunk Distribution of OpenTelemetry .NET in the following ways:
For .NET applications, set environment variables. On Windows, set them in the process scope unless you want to activate automatic instrumentation globally for all .NET applications.
For .NET Framework applications running as Windows services, you can add settings in the
appSettingsblock of the app.config file when supported or set environment variables using the Windows Registry.For ASP.NET applications, add settings in the
appSettingsblock of the web.config file. For example:<configuration> <appSettings> <add key="OTEL_SERVICE_NAME" value="my-service-name" /> </appSettings> </configuration>Alternatively, you can set environment variables using any of the following methods:
Add the
<environmentVariables>element inapplicationHost.configfor your application pools.Set the environment variables for
W3SVCandWAS.
For ASP.NET Core applications, add
<environmentVariable>elements inside the<aspNetCore>block of your web.config file.
General settings
The following settings are common to most instrumentation scenarios:
| Description | Environment Variable | Default Value | Type |
|---|---|---|---|
| Names of the executable files that you don't want the profiler to instrument. Supports multiple semicolon-separated values, for example: `ReservedProcess.exe;powershell.exe`. Notice that applications launched using dotnet MyApp.dll have process name `dotnet` or `dotnet.exe`. Can't be set using the web.config or app.config files. | OTEL_DOTNET_AUTO_EXCLUDE_PROCESSES | (empty) | string |
| Long Traces are collected by default. To deactivate trace collection, set the environment variable to `false`. Data from custom or manual instrumentation is not affected. | OTEL_DOTNET_AUTO_TRACES_ENABLED | true | boolean |
| Metrics are collected by default. To deactivate metric collection, set the environment variable to `false`. Data from custom or manual instrumentation is not affected. | OTEL_DOTNET_AUTO_METRICS_ENABLED | true | boolean |
| Logs are collected by default. To deactivate log collection, set the environment variable to `false`. Data from custom or manual instrumentation is not affected. | OTEL_DOTNET_AUTO_LOGS_ENABLED | true | boolean |
| Activates the OpenTracing tracer. The default value is `false`. | OTEL_DOTNET_AUTO_OPENTRACING_ENABLED | false | boolean |
| Activates immediate redirection of the assemblies used by the automatic instrumentation on the .NET Framework. The default values is `true`. Can't be set using the web.config or app.config files. | OTEL_DOTNET_AUTO_NETFX_REDIRECT_ENABLED | true | boolean |
| Controls whether the telemetry data is flushed when an `AppDomain.UnhandledException` event is raised. Set to `true` when experiencing missing telemetry at the same time of unhandled exceptions. | OTEL_DOTNET_AUTO_FLUSH_ON_UNHANDLEDEXCEPTION | false | boolean |
| Activates RuleEngine. The default values is `true`. RuleEngine increases the stability of the instrumentation by validating assemblies for unsupported scenarios. | OTEL_DOTNET_AUTO_RULE_ENGINE_ENABLED | true | boolean |
| Activate to let the process fail when automatic instrumentation can't be executed. This setting is for debugging purposes, don't use it in production environments. The default value is `false`. Can't be set using the web.config or app.config files. | OTEL_DOTNET_AUTO_FAIL_FAST_ENABLED | false | boolean |
Exporter settings
The following settings control trace exporters and their endpoints:
| Description | Environment variable | Default value | Type |
|---|---|---|---|
| Comma-separated list of exporters. Supported options: `otlp`, `console`, `none`. | OTEL_LOGS_EXPORTER | otlp | string |
| Comma-separated list of exporters. Supported options: `otlp`, `console`, `none`. | OTEL_METRICS_EXPORTER | otlp | string |
| Comma-separated list of exporters. Supported options: `otlp`, `console`, `none`. | OTEL_TRACES_EXPORTER | otlp | string |
| The URL to where traces, metrics, and logs are sent. The default value is `http://localhost:4318`. Setting a value overrides the `SPLUNK_REALM` environment variable. | OTEL_EXPORTER_OTLP_ENDPOINT | (empty) | string |
| Equivalent to `OTEL_EXPORTER_OTLP_ENDPOINT`, but applies only to logs. | OTEL_EXPORTER_OTLP_LOGS_ENDPOINT | (empty) | string |
| Equivalent to `OTEL_EXPORTER_OTLP_ENDPOINT`, but applies only to metrics. | OTEL_EXPORTER_OTLP_METRICS_ENDPOINT | (empty) | string |
| Equivalent to `OTEL_EXPORTER_OTLP_ENDPOINT`, but applies only to traces. | OTEL_EXPORTER_OTLP_TRACES_ENDPOINT | (empty) | string |
| The name of your organization’s realm, for example, `us0`. When you set the realm, telemetry is sent directly to the ingest endpoint of Splunk Observability Cloud, bypassing the Splunk Distribution of OpenTelemetry Collector. | SPLUNK_REALM | (empty) | string |
| A Splunk authentication token that lets exporters send data directly to Splunk Observability Cloud. Unset by default. Required if you need to send data to the Splunk Observability Cloud ingest endpoint. | SPLUNK_ACCESS_TOKEN | (empty) | string |
| The aggregation temporality to use on the basis of instrument kind. The supported options are `Cumulative` for all instrument kinds and `Delta` for Counter, Asynchronous Counter, and Histogram instrument kinds. If you use `Delta` for UpDownCounter and Asynchronous UpDownCounter instrument kinds, the `Cumulative` aggregation temporality will be used. `LowMemory`, from the OpenTelemetry specification, is not supported. | OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE | Cumulative | string |
.NET OTel settings for AlwaysOn Profiling
The following settings control the AlwaysOn Profiling feature for the .NET instrumentation:
| Description | Environment variable | Default value | Type |
|---|---|---|---|
| Activates AlwaysOn Profiling. | SPLUNK_PROFILER_ENABLED | false | boolean |
| Activates memory profiling. | SPLUNK_PROFILER_MEMORY_ENABLED | false | boolean |
| The collector endpoint for profiler logs. | SPLUNK_PROFILER_LOGS_ENDPOINT | http://localhost:4318/v1/logs | string |
| Frequency with which call stacks are sampled, in milliseconds. | SPLUNK_PROFILER_CALL_STACK_INTERVAL | 10000 | int |
Trace propagation settings
The following settings control trace propagation:
| Description | Environment variable | Default value | Type |
|---|---|---|---|
| Comma-separated list of propagators for the tracer. The default value is `tracecontext,baggage`. Supported values are `b3multi`, `b3`, `tracecontext`, and `baggage`. | OTEL_PROPAGATORS | tracecontext,baggage | string |
Samplers configuration
The following settings control trace sampling:
| Description | Environment variable | Default value | Type |
|---|---|---|---|
| Sampler to use. The default value is `parentbased_always_on`. Supported values are `always_on`, `always_off`, `traceidratio`, `parentbased_always_on`, `parentbased_always_off`, and `parentbased_traceidratio`. | OTEL_TRACES_SAMPLER | parentbased_always_on | string |
| Semicolon-separated list of rules for the `rules` sampler. The default value is `1.0` for `traceidratio`. | OTEL_TRACES_SAMPLER_ARG | 1.0 | string |
Resource detectors configuration
You can use resource detectors to retrieve additional attributes for your application’s spans.
| Description | Environment variable | Default value | Type |
|---|---|---|---|
| Activates or deactivates all resource detectors. The default values is `true`. | OTEL_DOTNET_AUTO_RESOURCE_DETECTOR_ENABLED | true | boolean |
| Activates or deactivates a specific resource detector, where `{DETECTOR}` is the uppercase identifier of the resource detector you want to activate. Overrides `OTEL_DOTNET_AUTO_RESOURCE_DETECTOR_ENABLED`. | OTEL_DOTNET_AUTO_{DECTECTOR}_RESOURCE_DETECTOR_ENABLED | depends on `OTEL_DOTNET_AUTO_RESOURCE_DETECTOR_ENABLED` | boolean |
Resource attributes
Resource attributes let you attach custom metadata to every span reported by your application.
| Description | Environment variable | Default value | Type |
|---|---|---|---|
| Name of the service or application you're instrumenting. Takes precedence over the service name defined in the `OTEL_RESOURCE_ATTRIBUTES` variable. | OTEL_SERVICE_NAME | (empty) | string |
| Comma-separated list of resource attributes added to every reported span. For example, `key1=val1,key2=val2`. | OTEL_RESOURCE_ATTRIBUTES | (empty) | string |
Instrumentation settings
The following settings control instrumentations and tracing behavior:
| Description | Environment variable | Default value | Type |
|---|---|---|---|
| Activated by default. Adds server trace information to HTTP response headers. The default value is `true`. | SPLUNK_TRACE_RESPONSE_HEADER_ENABLED | true | boolean |
| Comma-separated list of additional `System.Diagnostics.Metrics.Meter` names to be added to the meter at the startup. Use it to capture custom metrics. | OTEL_DOTNET_AUTO_METRICS_ADDITIONAL_SOURCES | (empty) | string |
| Maximum number of attributes per span. Default value is unlimited. | OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT | (empty) | int |
| Maximum number of events per span. Default value is unlimited. | OTEL_SPAN_EVENT_COUNT_LIMIT | (empty) | int |
| Maximum number of links per span. Default value is `1000`. | OTEL_SPAN_LINK_COUNT_LIMIT | 1000 | int |
| Maximum length of strings for attribute values. Values larger than the limit are truncated. Default value is `1200`. Empty values are treated as infinity. | OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT | 1200 | int |
| Comma-separated list of additional legacy source names to be added to the tracer at the startup. Use it to capture `System.Diagnostics.Activity` objects created without using the `System.Diagnostics.ActivitySource` API. | OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_LEGACY_SOURCES | (empty) | string |
| Activates or deactivates all instrumentations. Can’t be set using the web.config or app.config files. | OTEL_DOTNET_AUTO_INSTRUMENTATION_ENABLED | true | boolean |
| Activates or deactivates all trace instrumentations. Overrides `OTEL_DOTNET_AUTO_INSTRUMENTATION_ENABLED`. Inherits the value of the `OTEL_DOTNET_AUTO_INSTRUMENTATION_ENABLED` environment variable. Can’t be set using the web.config or app.config files. | OTEL_DOTNET_AUTO_TRACES_INSTRUMENTATION_ENABLED | (empty) | boolean |
| Activates or deactivates a specific trace instrumentation, where `{INSTRUMENTATION}` is the case-sensitive name of the instrumentation. Overrides `OTEL_DOTNET_AUTO_TRACES_INSTRUMENTATION_ENABLED`. Inherits the value of the `OTEL_DOTNET_AUTO_TRACES_INSTRUMENTATION_ENABLED` environment variable. Can’t be set using the web.config or app.config files. See Supported libraries for a complete list of supported instrumentations and their names. | OTEL_DOTNET_AUTO_TRACES_{INSTRUMENTATION}_INSTRUMENTATION_ENABLED | (empty) | boolean |
| Activates or deactivates all metric instrumentations. Overrides `OTEL_DOTNET_AUTO_INSTRUMENTATION_ENABLED`. Inherits the value of the `OTEL_DOTNET_AUTO_INSTRUMENTATION_ENABLED` environment variable. Can’t be set using the web.config or app.config files. | OTEL_DOTNET_AUTO_METRICS_INSTRUMENTATION_ENABLED | (empty) | boolean |
| Activates or deactivates a specific metric instrumentation, where `{INSTRUMENTATION}` is the case-sensitive name of the instrumentation. Overrides `OTEL_DOTNET_AUTO_METRICS_INSTRUMENTATION_ENABLED`. Inherits the value of the `OTEL_DOTNET_AUTO_METRICS_INSTRUMENTATION_ENABLED` environment variable. Can’t be set using the web.config or app.config files. See Supported libraries for a complete list of supported instrumentations and their names. | OTEL_DOTNET_AUTO_METRICS_{INSTRUMENTATION}_INSTRUMENTATION_ENABLED | (empty) | boolean |
| Activates or deactivates all log instrumentations. Overrides `OTEL_DOTNET_AUTO_INSTRUMENTATION_ENABLED`. Inherits the value of the `OTEL_DOTNET_AUTO_INSTRUMENTATION_ENABLED` environment variable. Can’t be set using the web.config or app.config files. | OTEL_DOTNET_AUTO_LOGS_INSTRUMENTATION_ENABLED | (empty) | boolean |
| Activates or deactivates a specific log instrumentation, where `{INSTRUMENTATION}` is the case-sensitive name of the instrumentation. Overrides `OTEL_DOTNET_AUTO_LOGS_INSTRUMENTATION_ENABLED`. Inherits the value of the `OTEL_DOTNET_AUTO_LOGS_INSTRUMENTATION_ENABLED` environment variable. Can’t be set using the web.config or app.config files. See Supported libraries for a complete list of supported instrumentations and their names. | OTEL_DOTNET_AUTO_LOGS_{INSTRUMENTATION}_INSTRUMENTATION_ENABLED | (empty) | boolean |
Diagnostic logging settings
The following settings control the internal logging of the Splunk Distribution of OpenTelemetry .NET:
| Description | Environment variable | Default value | Type |
|---|---|---|---|
| AutoInstrumentation diagnostic logs sink. (supported values: `none`,`file`,`console`. | OTEL_DOTNET_AUTO_LOGGER | file | string |
| Sets the logging level for instrumentation log messages. Possible values are `none`, `error`, `warn`, `info`, and `debug`. Can't be set using the web.config or app.config files. | OTEL_LOG_LEVEL | info | string |
| Directory of the .NET tracer logs. The default value is `/var/log/opentelemetry/dotnet` for Linux, and `%ProgramData%\OpenTelemetry .NET AutoInstrumentation\logs` for Windows. Can't be set using the web.config or app.config files. | OTEL_DOTNET_AUTO_LOG_DIRECTORY | (empty) | string |
| Deprecated. Whether the traces console exporter is activated. It can be configured by `OTEL_TRACES_EXPORTER`. | OTEL_DOTNET_AUTO_TRACES_CONSOLE_EXPORTER_ENABLED | false | boolean |
Changing the default service name
By default, the Splunk Distribution of OpenTelemetry .NET retrieves the service name by trying the following steps until it succeeds:
The default service name is the name of the entry assembly. For example, the name of your .NET project file. For ASP.NET applications, the default service name is
SiteName[/VirtualPath].If the entry assembly is not available, the instrumentation tries to use the current process name. The process name can be
dotnetif launched directly using an assembly. For example,dotnet InstrumentedApp.dll.
If all the steps fail, the service name defaults to unknown_service.
OTEL_SERVICE_NAME environment variable.Environment variables for manual installation
When deploying the instrumentation manually, you need to make sure to set the following environment variables:
- Windows (.NET)
Environment variable
Value
CORECLR_ENABLE_PROFILING1CORECLR_PROFILER{918728DD-259F-4A6A-AC2B-B85E1B658318}CORECLR_PROFILER_PATH_64$installationLocation\win-x64\OpenTelemetry.AutoInstrumentation.Native.dllCORECLR_PROFILER_PATH_32$installationLocation\win-x86\OpenTelemetry.AutoInstrumentation.Native.dllDOTNET_ADDITIONAL_DEPS$installationLocation\AdditionalDepsDOTNET_SHARED_STORE$installationLocation\storeDOTNET_STARTUP_HOOKS$installationLocation\net\OpenTelemetry.AutoInstrumentation.StartupHook.dllOTEL_DOTNET_AUTO_HOME$installationLocationOTEL_DOTNET_AUTO_PLUGINSSplunk.OpenTelemetry.AutoInstrumentation.Plugin, Splunk.OpenTelemetry.AutoInstrumentation- Windows (.NET Framework)
Environment variable
Value
COR_ENABLE_PROFILING1COR_PROFILER{918728DD-259F-4A6A-AC2B-B85E1B658318}COR_PROFILER_PATH_64$installationLocation\win-x64\OpenTelemetry.AutoInstrumentation.Native.dllCOR_PROFILER_PATH_32$installationLocation\win-x86\OpenTelemetry.AutoInstrumentation.Native.dllOTEL_DOTNET_AUTO_HOME$installationLocationOTEL_DOTNET_AUTO_PLUGINSSplunk.OpenTelemetry.AutoInstrumentation.Plugin, Splunk.OpenTelemetry.AutoInstrumentation- Linux (.NET)
Environment variable
Value
CORECLR_ENABLE_PROFILING1CORECLR_PROFILER{918728DD-259F-4A6A-AC2B-B85E1B658318}CORECLR_PROFILER_PATH$INSTALL_DIR/linux-x64/OpenTelemetry.AutoInstrumentation.Native.so(glibc)$INSTALL_DIR/linux-musl-x64/OpenTelemetry.AutoInstrumentation.Native.so(musl)DOTNET_ADDITIONAL_DEPS$INSTALL_DIR\AdditionalDepsDOTNET_SHARED_STORE$INSTALL_DIR\storeDOTNET_STARTUP_HOOKS$INSTALL_DIR\net\OpenTelemetry.AutoInstrumentation.StartupHook.dllOTEL_DOTNET_AUTO_HOME$INSTALL_DIROTEL_DOTNET_AUTO_PLUGINSSplunk.OpenTelemetry.AutoInstrumentation.Plugin, Splunk.OpenTelemetry.AutoInstrumentationNote: The default installation path on Linux is$HOME/.otel-dotnet-auto.