Configure Splunk indexing and forwarding to use TLS certificates

Before you configure TLS certificates for indexing and forwarding, you must have the following:

• One or more TLS certificates in Privacy-Enhanced Mail (PEM) format that comply with the x.509 public key certificate standard. You can either obtain third party certificates from a certificate authority, or create and sign them yourself
• A private key in PEM format that matches the certificate
• Administrative access to the Splunk Enterprise instances that you want to secure

You can configure indexers to use TLS certificates. The certificates you configure on indexers control how the indexer receives data from a forwarder.

When you configure Splunk Enterprise to use TLS certificates, upon restart, it changes the file permissions on the certificates so that only the user that Splunk Enterprise runs as has full access. This is by design, in line with security industry standards, and cannot be changed.

1. Open a shell or command prompt.
2. Using this prompt or file system management tools, copy the server certificate and the certificate authority public certificate into an accessible directory on the indexer where you want to configure certificates. For example, you can move the files to a destination directory of $SPLUNK_HOME/etc/auth/mycerts/.
3. Use a text editor to open the $SPLUNK_HOME/etc/system/local/inputs.conf configuration file for editing.
4. In the inputs.conf file, configure the indexer to use the server certificate. Add the following stanzas and settings to the file.

   Setting or stanza	Data type	Description
   [splunktcp-ssl:port]	stanza	Defines a TCP network input to receive data over TLS on the port you specify.
   [SSL]	stanza	Defines the TLS settings for all inputs defined for this instance.
   serverCert	string	The location of the server certificate on the Splunk platform instance. You can specify an absolute path, such as /opt/splunk/etc/auth/mycerts/myServerCert.pem, or a path relative to $SPLUNK_HOME, such as etc/auth/mycerts/myServerCert.pem.
   sslPassword (Optional)	string	The password for the private key, if you created one. Do not configure this setting if you did not specify a password when you created your private key.
   requireClientCert (Optional)	Boolean	Whether or not the indexer must receive a valid client certificate. A value of "true" means that the indexer must see a valid client certificate from the connecting forwarder. It also means that mutually authenticated TLS (mTLS) is active.
   sslVersions (Optional)	comma-separated list	The list of TLS versions that the receiver supports. Valid values are "tls1.2" and "tls1.3". The default at version 10.4.0 and higher is tls1.2,tls1.3. SSLv2 and SSLv3 are always turned off. TLS 1.0 and TLS 1.1 are turned off by default in version 10.4.0 and higher.
   cipherSuite (Optional)	string	The cipher suite strings that TLS 1.2 sessions are to use. This setting does not apply to TLS 1.3 connections. To configure TLS 1.3 cipher suites, use the [tls1.3] stanza in the server.conf configuration file.
   sslCommonNameToCheck (Optional)	comma-separated list	A list of one or more common names, or fully-qualified host names, upon which the indexer checks for a match in the certificate that the client presents upon connecting to the receiver. This setting is only valid if you have configured the requireClientCert setting with a value of "true". If none of the common names in this setting value matches the common name in the certificate of the connecting client, the indexer declines the connection as not authorized.
   sslAltNameToCheck (Optional)	comma-separated list	A list of one or more alternate names upon which the indexer checks for a match in the certificate that the client presents upon connecting to the receiver. This setting is only valid if you have configured the requireClientCert setting with a value of "true". If none of the alternate names in this setting value matches the alternate name in the certificate of the connecting client, the indexer declines the connection as not authorized.

5. Save the inputs.conf file and close it.
6. Open the $SPLUNK_HOME/etc/system/local/server.conf configuration file for editing.
   Note: Machines that run Windows look in the Registry for certificate authority certificates by default.
7. Add the location of the certificate authority certificate to the [sslConfig] stanza:
   CODECopy

   [sslConfig] sslRootCAPath = <absolute path to the CA certificate>

   [sslConfig]
   sslRootCAPath = <absolute path to the CA certificate>

   Note: If you want to override where machines that run Windows look for certificate authority certificates, in addition to adding the sslRootCAPath entry in the previous step, add the entry sslRootCAPathHonoredOnWindows = true on its own line. While the setting is deprecated, it remains available for use when you want to control where the system looks for CA certificates on Windows.
8. Save the server.conf file and close it.
9. Reload the TLS configuration. TLS settings in inputs.conf are reloadable and do not require a full restart of splunkd. Reload each affected input using the Splunk REST API or the Splunk Web Settings interface.

The certificates you configure on forwarders control how the forwarder connects to the indexer and how it communicates with the indexer to send data. If you configure the indexer, or receiver, to require a client certificate, you must configure the forwarder to present that client certificate when it connects to the indexer.

1. Copy the new certificate and the certificate authority certificate files into an accessible folder on the forwarders you want to configure. For example, you can use a destination folder of $SPLUNK_HOME/etc/auth/mycerts on the forwarder.
2. Using a text editor, open the $SPLUNK_HOME/etc/system/local/outputs.conf file for editing.
3. Use the following settings to define the [tcpout] stanza in the file to configure the forwarder to use the certificate.

   Setting or stanza	Data type	Description
   [tcpout:name]	stanza	Defines an output group to send data to a receiver.
   server	string	The hostname or IP address and port on which to connect securely to forward data.
   clientCert	string	The location of the client certificate on the forwarder. This is the certificate that the forwarder uses to connect to the receiving indexer over TLS. You can specify an absolute path, such as /opt/splunk/etc/auth/mycerts/myClientCertificate.pem, or a path relative to $SPLUNK_HOME, such as etc/auth/mycerts/myClientCertificate.pem.
   useClientSSLCompression (Optional)	Boolean	Whether or not the forwarder performs TLS compression when it connects with a receiver. The default value of "true" means that the client uses TLS compression. A value of "false" means the client does not use compression. This setting has no effect for TLS 1.3 connections because TLS 1.3 does not support SSL compression.
   sslPassword (Optional)	string	Same as the setting in the inputs.conf configuration file.
   sslVerifyServerCert (Optional)	Boolean	Whether or not, upon connection to a receiver, the forwarder confirms that the receiver has a valid TLS server certificate. A value of "true" means that the forwarder checks for a valid server certificate upon connection, then checks the common or alternate names against the values for the sslCommonNameToCheck and sslAltNameToCheck settings on the forwarder. If there is no match against the common or alternate names, the forwarder aborts the connection to the receiver as not authorized.
   sslVerifyServerName (Optional)	Boolean	Whether or not, upon connection to a receiver, the forwarder confirms that the valid TLS certificate that the receiver presents contains the host name of the receiver in the common name or subject alternative name field of the certificate. A value of "true" means that the forwarder checks for a host name match in the certificate that the receiver presents. If the host name in the certificate does not match the receiver host name, the forwarder aborts the connection to the receiver as not authorized.
   sslCommonNameToCheck (Optional)	comma-separated list	Same as the setting in the inputs.conf configuration file, except that you must give the sslVerifyServerCert setting a value of "true" in the outputs.conf configuration file and the forwarder does the certificate verification.
   sslAltNameToCheck (Optional)	comma-separated list	Same as the setting in the inputs.conf configuration file, except that you must give the sslVerifyServerCert setting a value of "true" in the outputs.conf configuration file and the forwarder does the certificate verification.
   cipherSuite (Optional)	string	The TLS 1.2 cipher suite strings that the forwarder uses when connecting to a receiver. Separate multiple cipher suites with a colon (:). This setting does not apply to TLS 1.3 connections.

4. Save the outputs.conf file and close it.
5. On forwarders that do not run on Windows, open the server.conf configuration file for editing.
   Note: Machines that run Windows look in the Registry for certificate authority certificates by default.
6. Add the following stanza and settings to the file:
   CODECopy

   [sslConfig] sslRootCAPath = <absolute path to the certificate authority certificate>

   [sslConfig]
   sslRootCAPath = <absolute path to the certificate authority certificate>

   Note: If you want to override where machines that run Windows look for certificate authority certificates, in addition to adding the sslRootCAPath entry in the previous step, add the entry sslRootCAPathHonoredOnWindows = true on its own line. While the setting is deprecated, it remains available for use when you want to control where the system looks for CA certificates on Windows.
7. Save the server.conf file and close it.
8. Restart the splunkd process.
   CODECopy

   $SPLUNK_HOME/bin/splunk restart splunkd

   $SPLUNK_HOME/bin/splunk restart splunkd

To forward data securely to multiple indexers, add a server entry for each indexer to the output group definition stanza in the outputs.conf configuration file. Separate multiple entries with commas.

1. On the forwarder where you want to send data to multiple indexers, use the "Configure forwarders to use a signed certificate" procedure earlier in this topic to open and make changes to the outputs.conf configuration file.
2. In the target output group definition stanza for the forwarder, add a host:port line for each indexer to which you want to send data over TLS. Separate multiple entries with commas.
3. Save the outputs.conf file and close it.
4. Restart the forwarder.

The following example outputs.conf file uses the same certificate for the forwarders as it does the indexers:

[tcpout]

[tcpout:group1]
server = 10.1.12.112:9997,10.1.12.111:9999
disabled = 0
clientCert = $SPLUNK_HOME/etc/auth/client.pem
sslPassword = <password for the client certificate>
sslCommonNameToCheck = indexercn.example.org
sslVerifyServerCert = true

If you have created one server certificate for each indexer and you have set a unique sslCommonNameToCheck or sslAltNameToCheck in each indexer certificate to be checked by the forwarders, you must configure one [tcpout-server://host:port] configuration stanza for each indexer in the outputs.conf file. This action lets you specify which name to check for each indexer.

Confirm that the forwarder and indexer configurations work properly. See Test and troubleshoot TLS connections.