Inspect a Machine Data Lake (MDL) raw table and update its management settings or data landing definition.
-
You have an existing Machine Data Lake (MDL) raw table and permission to view or edit the dataset.
-
To edit the underlying Ingest Processor pipeline directly, your role must include the required Ingest Processor management permissions, such as admin_all_objects where that capability is required. Machine Data Lake does not require a separate Machine Data Lake-specific capability for this access.
After you create a raw table, you can inspect dataset details, manage dataset settings, edit the data landing definition, promote data, search data, or delete the dataset.
Landing pipelines use Ingest Processor resources in the background and count toward applicable Ingest Processor limits together with streaming promotion pipelines.
- From the global navigation bar in Splunk Cloud Platform, select the Catalog (
) icon.
On the Catalog page, the Datasets tab is selected by default.
- Select the raw table that you want to inspect or edit.
Use the search field to filter the dataset list. The dataset details panel shows general dataset information, event summary, field summary, and promotion jobs.
- Open the raw table details page.
From the dataset details panel, select the more icon, and then select Manage. The raw table details page opens.
- Choose an action for the raw table.
- To promote data from the raw table, select Promote.
- To search the raw table, select Search. Searching raw tables can use more compute than searching promoted data, and only a limited set of fields is searchable. For better search performance, promote the data before you search.
- To delete the dataset, select Delete. Confirm the deletion when prompted.
- To review dataset metadata and related promotion jobs, select the Overview tab.
- To update dataset management settings, select the Manage tab.
- To update the data landing definition, select the Edit data landing tab.
- (Optional) On the Overview tab, review the raw table details.
The Overview tab shows general information, data retention, fields, and related promotion jobs. To refresh the list of related promotion jobs, select Refresh jobs.
- (Optional) On the Manage tab, update the dataset management settings.
You can update Open Sharing settings, dataset description, data retention, promotion limits, and access control settings.
- (Optional) In the External sharing section, select an expiration period, and then select Enable sharing.
Open Sharing makes the dataset available as read-only, zero-copy data by using Delta Sharing. For more information, see Read Delta Sharing Tables.
- (Optional) In Dataset description, enter or update the dataset description.
- In Data retention for raw table, enter how long to keep data in the raw table, and then select the time unit.
- In the Promotion limits section, enter the maximum rolling window for streaming promotions, and then select the time unit.
Data outside the rolling window is removed from the promoted destination. This setting doesn't affect the source raw table retention period.
- Enter the default retention period for static promotions, and then select the time unit.
This setting controls the retention period for all promotion targets created by static promotion created from the raw table.
- (Optional) In the Access control section, add roles or accounts that need dataset-specific access.
Select one or more roles or accounts from Add roles and users, select the add icon, and then select an access level for each role or account. Default access control is inherited from capabilities and policies.
- Select Save.
The raw table management settings are updated.
- (Optional) On the Edit data landing tab, update the data landing definition.
You can add partitions, add sample data, add actions, review fields, preview the landing definition, or turn on SPL editor, the Search Processing Language (SPL) editor, to view or edit the landing definition as Search Processing Language version 2 (SPL2). Partitions and actions affect preview results and the data written to the raw table after the landing definition is saved.
Use Edit data landing to change which events from an existing supported HEC, Universal Forwarder, or Heavy Forwarder path to a Splunk index land in this raw table. Before changing routing, record the current routing selection and landing definition. Identify any searches, dashboards, or alerts that depend on the existing index. If matching events must continue to that index, confirm that events already reach it. Preview representative matching and non-matching events to check the partition conditions and processing actions before saving.
- (Optional) In the Partition section, add or update partitions by sourcetype, source, or host.
Use the IN operator and enter or select at least one non-empty value for each partition. Data that does not match the partition conditions follows the configured non-matching destination for the pipeline. For more information, see How data moves through the Ingest Processor solution.
- In the process_data panel, under Partition, open the routing drop-down and select an option.
-
Divert data to raw table: Send matching events only to this raw table instead of the existing Splunk index.
-
Duplicate data to raw table: Send matching events to this raw table and continue sending them to the existing Splunk index.
Events that do not match the partition conditions continue to their configured destination.
This setting applies to new incoming events after the saved landing definition takes effect. It does not copy historical data from the existing index into Machine Data Lake.
Diverting matching events can affect searches, dashboards, and alerts that depend on the existing index. Duplicating events retains data in both destinations. Review Machine Data Lake limits and caveats when planning capacity.
- (Optional) Select Add sample data, enter or upload representative sample data, and then select Apply.
Use sample data in the same format as the events that you want to land. Include events with the source, sourcetype, and host values that you use in the landing definition. For more information, see Create pipelines for Ingest Processor.
- (Optional) In the Actions section, add or update actions that filter, mask, or process matching events.
- Select Preview to confirm that the results match the events that you want to land.
If you change partitions, actions, or the Search Processing Language version 2 definition, run preview again to confirm the results before you save.
- Select Save.
- Verify delivery of new incoming events.
After the saved landing definition takes effect, send a uniquely identifiable matching test event that the landing actions retain. Use its source, sourcetype, host, and time to locate and inspect it.
-
For Duplicate data to raw table, verify that the event reaches both the raw table and the existing index.
-
For Divert data to raw table, verify that the event reaches the raw table and is not sent to the existing index through this route.
Verify that a representative non-matching event continues to its configured destination. Preview checks the landing definition. Verify destination delivery after saving.
The raw table landing definition is updated. New incoming events follow the saved routing selection.