Troubleshoot log collection

Check Collector logs

Use the following commands to check Collector logs:

Linux

To follow Collector logs in real time, run the following command:

BASHCopy

sudo journalctl -u splunk-otel-collector -f

sudo journalctl -u splunk-otel-collector -f

To view the last 100 lines, run the following command:

BASHCopy

sudo journalctl -u splunk-otel-collector -n 100

sudo journalctl -u splunk-otel-collector -n 100

Windows

To view the last 100 Collector events, run the following command in PowerShell:

POWERSHELLCopy

Get-WinEvent -FilterHashtable @{ LogName='Application'; ProviderName='splunk-otel-collector' } -MaxEvents 100

Get-WinEvent -FilterHashtable @{ LogName='Application'; ProviderName='splunk-otel-collector' } -MaxEvents 100

Common errors

Use the following sections to troubleshoot common errors from Linux and Windows log collection.

TLS error: x509 can't validate certificate for IP SANs

This error occurs when the HEC endpoint URL uses a raw IP address, such as https://10.x.x.x:8088/services/collector, but the server TLS certificate was issued for a hostname instead of an IP address. To resolve this issue, use the hostname in --splunk-platform-url.

If the server uses a self-signed certificate or an internal certificate authority (CA), add the CA to the system trust store on the Collector host so the certificate can be verified.

As a last resort in non-production environments, you can turn off TLS verification:

exporters:
  splunk_hec/logs:
    tls:
      insecure_skip_verify: true

Exporting failed: context deadline exceeded

This error means the HEC endpoint accepted the connection but didn't respond before the client timeout expired. It often happens when the Splunk platform is under heavy load and can't process incoming data fast enough. The exporter retries requests. Data remains queued while queue capacity is available.

To give the Splunk platform more time to respond, increase the HTTP client timeout in splunk_logs_config_linux.yaml or splunk_logs_config_windows.yaml:

exporters:
  splunk_hec/logs:
    timeout: 30s

If retries fill the queue and cause backpressure, reduce log volume or deactivate high-traffic receivers.

No logs arrive in the Splunk platform

Use the following steps to troubleshoot missing logs for your operating system:

Linux

1. Check that the Collector is running:

   BASHCopy

   sudo systemctl status splunk-otel-collector

   sudo systemctl status splunk-otel-collector

2. Verify that the HEC token and URL are correct in /etc/otel/collector/splunk-otel-collector.conf.

3. Confirm that the receiver is enabled in the pipeline. Check service.pipelines.logs/hec.receivers in the configuration.

4. Look for export errors in the Collector logs.

5. Check for common HEC errors, such as 401 Unauthorized for a bad token, 403 Forbidden for a deactivated token or wrong index, or 400 Bad Request for a malformed event.

Windows

1. Check that the Collector service is running:

   POWERSHELLCopy

   Get-Service -Name splunk-otel-collector

   Get-Service -Name splunk-otel-collector

2. Verify that the HEC token and URL are correct in the Collector service environment:

   POWERSHELLCopy

   (Get-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\splunk-otel-collector').Environment

   (Get-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Services\splunk-otel-collector').Environment

3. Confirm that the receiver is enabled in the pipeline. Check service.pipelines in C:\ProgramData\Splunk\OpenTelemetry Collector\splunk_logs_config_windows.yaml.

4. Look for export errors in the Collector logs.

5. Check for common HEC errors, such as 401 Unauthorized for a bad token, 403 Forbidden for a deactivated token or wrong index, or 400 Bad Request for a malformed event.

6. Restart the Collector service after making configuration changes:

   POWERSHELLCopy

   Restart-Service splunk-otel-collector

   Restart-Service splunk-otel-collector

The journald receiver produces no data

The Collector log shows the following error when the service user doesn't have systemd-journal group membership:

Jun 02 09:41:29 ip-10-236-24-112 otelcol[103201]: 2026-06-02T09:41:29.462Z error journald/input.go:98 journalctl command exited {"otelcol.component.id": "journald", "otelcol.component.kind": "receiver", "error": "exit status 1"}

Verify group membership:

groups splunk-otel-collector

If systemd-journal isn't listed, run the following commands:

sudo usermod -aG systemd-journal splunk-otel-collector
sudo systemctl restart splunk-otel-collector

Memory usage is above the soft limit

This log message means the memory_limiter processor is rejecting incoming data because memory usage exceeded the soft limit. The soft limit is derived from SPLUNK_MEMORY_LIMIT_MIB, which is 90% of SPLUNK_MEMORY_TOTAL_MIB.

To resolve the issue, try the following options:

• Reduce the number of active receivers.

• Lower queue_size on the Splunk HEC exporter.

• Increase SPLUNK_MEMORY_TOTAL_MIB from the default 512 MB when your use case requires more memory. Modify splunk-otel-collector.conf and restart the Collector service.

If increasing SPLUNK_MEMORY_LIMIT_MIB doesn't resolve the issue, the Collector might be falling behind because of a throughput bottleneck, such as HEC or CPU capacity.