Linux kernel memory overcommitting and Splunk crashes
Many distributions of Linux have a parameter that controls how the kernel handles requests for large amounts of memory. The Memory Overcommit tunable kernel parameter (vm.overcommit_memory) determines whether the kernel accepts or denies such requests.
The parameter can be set by running a command such as sysctl or by echoing a value to a specific file on the machine. The parameter takes three values:
| Value | Default | Meaning |
|---|---|---|
| 0 | X | Heuristic overcommit handling, denying blatantly invalid requests |
| 1 | No overcommit handling | |
| 2 | Denies requests that are equal to or greater than the sum of available swap space and the percentage of physical memory as specified by the overcommit_ratio parameter
|
How this setting impacts Splunk performance
Setting this parameter to anything other than the default of 0 can cause Splunk Enterprise to crash frequently with logs that do not reliably provide information on the root cause of the crash. In particular, setting the overcommit_memory parameter to 2 can result in crashes that provide no information, which makes troubleshooting and problem diagnosis difficult to impossible.
Confirm that you have set the vm.overcommit_memory parameter to its default of 0, unless you run an app on the same machine that requires changing this parameter. As a best practice, do not run other applications on a machine that runs Splunk Enterprise, unless those applications are directly related to or required by Splunk services.
See the following for additional information about the memory overcommitting feature:
- Capacity Tuning on the Red Hat Linux Performance Tuning Guide
- Your Linux distribution documentation on memory management