Configure Linux systems running systemd
If your Linux system is running systemd, you must configure systemd to manage splunkd as a service by creating a unit file that defines a cgroup hierarchy.
The following diagram illustrates the cgroup hierarchy under systemd:
For more information, see cgroups.
Configure systemd to manage splunkd as a service
There are two ways to configure systemd to manage splunkd as a service:
- Configure
systemdmanually. - Configure
systemdautomatically using thesplunk enable boot-startcommand.
systemd using splunk enable boot-start requires Splunk Enterprise version 7.2.2 or later.System requirements
To enable workload management in Splunk Enterprise under systemd requires systemd version 219 or higher. For more information, see Linux operating system requirements.
Permissions requirements for systemd
systemd has the following permissions requirements:
- Non-root users must have super user permissions to manually configure
systemdon Linux. - Non-root users must have super user permissions to run
start,stop, andrestartcommands undersystemd.
You must use sudo to run splunk start|stop|restart. If you do not use sudo, you must authenticate. For example:
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to manage system services or units.
Multiple identities can be used for authentication:
1. <username_1>
2. <username_2>
Choose identity to authenticate as (1-2): 2
Password:
==== AUTHENTICATION COMPLETE ===
Alternatively, you can install polkit rules to allow non-root users to run start, stop, and restart operations under systemd without using sudo, when running the enable-boot-start command to configure the systemd service automatically. For more information, see Install polkit rules to elevate user permissions.
Configure systemd manually
Follow these steps to configure systemd to manage splunkd as a service:
-
Confirm that your Linux machine is running
systemd. See Is Linux running systemd?. -
Before you create, delete, or modify the
systemdunit file, you must stopsplunkd.$SPLUNK_HOME/bin/splunk stop -
If you enabled Splunk software to start at boot using
enable boot-start, rundisable boot-startto remove thesplunkinitscript from/etc/init.dand its symbolic links.sudo $SPLUNK_HOME/bin/splunk disable boot-start -
Open the
$SPLUNK_HOME/etc/splunk-launch.conffile and note the value ofSPLUNK_SERVER_NAME. The default value isSplunkd. -
In the
/etc/systemd/systemdirectory, create a unit file (text file) named<SPLUNK_SERVER_NAME>.service, such asSplunkd.service.Note: You can change theSPLUNK_SERVER_NAMEto any name you choose by directly editing thesplunk-launch.conffile. -
Add the following content to the
<SPLUNK_SERVER_NAME>.serviceunit file:[Unit] After=network.target [Service] Type=simple Restart=always ExecStart=/opt/splunk/bin/splunk _internal_launch_under_systemd KillMode=mixed KillSignal=SIGINT TimeoutStopSec=360 LimitNOFILE=65536 SuccessExitStatus=51 52 RestartPreventExitStatus=51 RestartForceExitStatus=52 User=splunk Group=splunk Delegate=true CPUShares=1024 MemoryLimit=<value> PermissionsStartOnly=true ExecStartPost=/bin/bash -c "chown -R splunk:splunk /sys/fs/cgroup/cpu/system.slice/%n" ExecStartPost=/bin/bash -c "chown -R splunk:splunk /sys/fs/cgroup/memory/system.slice/%n" [Install] WantedBy=multi-user.targetNote: TheMemoryLimitvalue should be set to the total system memory available in bytes. TheMemoryLimitvalue will not update if the total available system memory changes. To update theMemoryLimitvalue in the unit file, manually edit the unit file value and run thesystemctl daemon-reloadcommand to reload systemd.CAUTION: The following unit file properties are set specifically for Splunk workload management:Type=simpleRestart=alwaysDelegate=trueDo not change these values unless you are familiar withsystemdor receive guidance from Splunk support.CAUTION: Do not use the following unit file properties. These properties can causesplunkdto fail on restart.RemainAfterExit=yesExecStopFor more information on
systemdunit file configuration settings, see Service unit configuration. -
Reload the unit file.
sudo systemctl daemon-reload -
Start
splunkd.sudo $SPLUNK_HOME/bin/splunk startThis starts
splunkdas asystemdservice.Note: Undersystemd,splunk start|stop|restartcommands are mapped tosystemctl start|stop|restartcommands. -
Verify that
splunkdis running as asystemdservice:sudo systemctl status <SPLUNK_SERVER_NAME>.serviceWhen you create the
splunkdservice,systemdcreates corresponding CPU and Memory cgroups in these locations:CPU: /sys/fs/cgroup/cpu/system.slice/<SPLUNK_SERVER_NAME>.service Memory: /sys/fs/cgroup/memory/system.slice/<SPLUNK_SERVER_NAME>.serviceNote: On Splunk Enterprise 8.0.0 or higher, on initial start of thesplunkdservice, the software checks thesystemdenvironment, identifies thecgrouppath, and automatically sets permissions for thecgroupdirectories. - For distributed deployments, repeat steps 1-9 on all search heads and indexers.
Manage clusters under systemd
When managing an indexer cluster under systemd:
You must use the sudo command to start, stop, and restart the cluster manager node or individual peer nodes using splunk start|stop|restart commands. You do not need sudo to perform a rolling restart using the splunk rolling-restart cluster-peers command, or to take a peer offline using the splunk offline command.
When managing a search head cluster under systemd:
You must use the sudo command to start, stop, and restart cluster members using splunk start|stop|restart commands. You do not need sudo to perform a rolling restart using the splunk rolling-restart shcluster-members command, or to remove a cluster member using the splunk remove shcluster-members command.
Upgrade considerations for systemd
Upgrade from 8.0.x to 8.1
If you configured Splunk Enterprise to run as a systemd service in version 8.0.x, upon upgrade to version 8.1, Splunk Enterprise automatically updates the existing Splunkd.service unit file with the following properties:
User
Group
ExecStartPost
Upgrade from 7.3.x or earlier to 8.0
If you configured Splunk Enterprise to run as a systemd service in version 7.3.x or lower, upon upgrade to version 8.0.0, Splunk Enterprise automatically updates the existing Splunkd.service unit file. For detailed information on these unit files changes, see Upgrade considerations for systemd in the Admin Manual.
Next step
After you set up cgroups on your Linux operating system, you can configure workload management in Splunk Enterprise. See Configure workload management.