Set up a data management control plane
The data management service is a management component that you can set up within an existing Splunk Enterprise deployment. It provides an on-premises control plane and accompanying UI that lets you deploy data management services to process and manage your data. Splunk Enterprise administrators who want to utilize data management services, such as the Edge Processor solution, within their Splunk Enterprise deployment must set up a data management control plane to host these services.
Where to install a data management control plane
You have options for where to host your data management control plane, depending on the nature of your deployment. The recommended best practice is to dedicate a separate machine and Splunk Enterprise instance as a host for your data management control plane.
A data management control plane can also be colocated with other management components or within a standalone Splunk instance. If you choose to colocate your control plane, ensure that the host machine has sufficient hardware specifications that can fully accommodate hosting both the control plane and the additional component or rest of your deployment. The host machine must also meet the software requirements. See Setup prerequisites for more information on hardware and software specifications, some of which will depend on the scope of the services that you plan to run on your control plane.
For a general discussion and disclaimers about management component colocation, see Components that help to manage your deployment in the Distributed Deployment Manual.
Setup prerequisites
Before setting up a data management control plane in your Splunk Enterprise deployment, ensure that the host machine where you're installing a new instance meets the following hardware, software, and network requirements. The following limitations also apply.
- The data management service is not supported on a search head cluster.
- FIPS mode is not supported and will automatically hide the data management service if detected on a Splunk Enterprise instance.
- Splunk Enterprise must be run as a non-root user. See Run Splunk Enterprise as a non-root user in the Installation Manual for more information.
- The data management service must be set up on a Splunk Enterprise instance running port 8000.
Hardware requirements
The host machine where you want to set up a data management control plane must meet the following requirements.
| Hardware | Specifications |
|---|---|
| CPUs | Dependent on the services run on the control plane. See the below table for tested Edge Processor specifications. |
| CPU architecture | x86 (64-bit) Note: Starting with Splunk Enterprise version 9.4, KV Store has prerequisites that must be met. The data management service depends on KV Store to store the Splunk tokens that it uses. Some virtualized environments may not be supported. See KV store server 7.0 pre-requisites in the Admin Manual for more information. |
| Memory | Dependent on the services run on the control plane. See the below table for tested Edge Processor specifications. |
| Storage | At least 50 GB. |
| Network | 1 Gb Ethernet NIC |
If you plan to deploy the Edge Processor service on your data management control plane, consider the following performance benchmarks on how many edge processors can be managed by a data management control plane on a dedicated machine with the corresponding specifications. For more information on scaling your Edge Processor service, see Add more instances to an Edge Processor.
| CPUs | Memory | Number of edge processors supported |
|---|---|---|
| 4 | 16 GB | 50 |
| 8 | 32 GB | 100 |
| 16 | 64 GB | 200 |
Software requirements
You can set up a data management control plane on a Splunk Enterprise instance that is version 10.0.0 and higher. A data management control plane does not require that other Splunk components also be run on the same operating system and version. Data management is available on all Linux operating systems supported for Splunk Enterprise. For a list of supported operating systems, see System requirements in the Installation Manual.
Network requirements
Configure your firewall settings and the ports on your host machines to allow communication between the control plane and the services you deploy.
| Scope | Port | Details |
|---|---|---|
| Localhost or 127.0.0.1 | 5432 | Data management depends on an instance of postgres included as part of the install. Postgres' default port is 5432. Make sure this is open for local loopback on the host machine. You do not need to expose this to external traffic. |
| Inbound | 8089 | Data management services need to listen and expose REST APIs. Splunk common practice is to use 8089. See Components and their relationship with the network in the Inherit a Splunk Deployment manual. |
Set up a data management control plane
Complete the following steps to set up a data management control plane in your Splunk Enterprise deployment.
- Log into the Splunk Enterprise instance where you want your data management control plane with admin credentials and using port 8000.
- Navigate to the Apps menu and select Data Management. You will be prompted to enable and configure available services to be hosted within your data management control plane.
- See First-time setup instructions for the Edge Processor solution for Edge Processor configurations.
- Fill out the corresponding fields for the services you want to deploy in your control plane.
- Select Save to save these settings to your .conf files.
- Once prompted, select Restart Splunk to restart your instance. You will be redirected to sign in.
- On the login screen, log in to your Splunk Enterprise instance using the same admin account.
- Navigate to the Data Management app and continue the setup process for your enabled services as needed.
Data management roles and capabilities
When you set up a data management control plane within your Splunk Enterprise deployment, new roles and their corresponding capabilities become available. These roles and capabilities can be assigned to users depending on the tasks the users will perform and manage in data management. The data management app adds three roles to the default roles provided by Splunk Enterprise, as seen in the table below.
| Role name | Description |
|---|---|
| admin | The default administrator. See Predefined Splunk Platform roles for more information. |
| data_management_admin | The primary admin user of the Data Management app. |
| data_management_agent | Role granted to a supervisor that manages the lifecycle of data management agents like Edge Processors and includes onboarding, monitoring, package and configuration updates. |
| data_management_agent_spl2 | Role used by the Edge Processor service to retrieve lookup values using SPL2 |
Each of these data management roles includes specific capabilities. The following table describes these capabilities and their corresponding roles.
| Capability name | What it lets you do | Roles with this capability |
|---|---|---|
| edit_data_management_pipeline | Lets a user access the data management pipelines API to perform creation, reading, updating, and deletion operations. |
|
| list_spl2_modules | Lets a user read and list SPL2 modules |
|
| edit_spl2_modules | Lets a user create, update, and delete SPL2 modules Note: Users with the data_management_admin role can create ingest destination datasets within the apps.pipeline_builders namespace.
|
|
| edit_spl2_datasets | Lets a user create, update, and delete datasets Note: Users with the data_management_admin role will can create SPL2 modules within the apps.pipeline_builders namespace.
|
|
| edit_data_management_pipeline_job | Lets a user deploy, update, and cancel the SPL2 ingest pipeline |
|
| run_spl2_search | Lets a user run and stop SPl2 search |
|
| preview_data_management_pipeline | Lets a user preview events in the SPL2 ingest pipeline. |
|
| edit_data_management_edgeprocessor | Lets a user manage edge processors. |
|
| provision_data_management_agent | Lets a user register data management agents. |
|
| edit_data_management_agent | Lets a user manage data management agents. |
|
| list_health | Lets a user see the Splunk health report.
Note: By default, only the admin role has this capability. If a user with a data_management_admin role would like to see the Splunk health report, the admin can add this capability to the data_management_admin role. |
|
For more information on capabilities and roles, see Define roles on the Splunk platform with capabilities in the Admin Manual.
Reference
- For information about the latest product updates that impact the data management control plane, see Data input issues in the Release Notes manual.
- The data management control plane runs on postgres and may require additional troubleshooting. See Troubleshoot data management for specific guidance on troubleshooting your data management control plane.