Prepare a data source for Machine Data Lake

Prepare a supported data source, choose how to route events to a Machine Data Lake raw table, and plan validation before changing production traffic.

1. Choose how the source routes matching events to Machine Data Lake.

   Use one of the following supported routes:

   • For application logs, cloud services, scripts, and integrations that send events through HTTP Event Collector (HEC), see Route HEC events directly to Machine Data Lake.

   • For supported logs and events collected through an OpenTelemetry (OTel) Collector pipeline or integration path, configure the supported export path that sends events to Splunk Cloud Platform. If the collector exports through HEC, use an HEC token that targets the raw table. See Splunk HEC exporter and Edit the Collector configuration to filter and send logs to Splunk Cloud Platform.

   • For files, operating system data, and server-generated logs collected by a Universal Forwarder, see Route Universal Forwarder data directly to Machine Data Lake.

   • For data that must be parsed, aggregated, or filtered by a Heavy Forwarder before it reaches Splunk Cloud Platform, see Route Heavy Forwarder data directly to Machine Data Lake.

   • For events that already reach Splunk Cloud Platform and must be selected without changing upstream clients, create or edit the raw table landing definition. See Create a Machine Data Lake raw table and Inspect and edit raw tables.

   For a conceptual summary of these routing options, see Choose how to route data to Machine Data Lake.

2. If matching events must also continue to a Splunk index, choose the dual-routing procedure for the selected route.

   • For HEC clients, send the same event separately with the Machine Data Lake HEC token and the HEC token for the Splunk index. See Send HEC events to Machine Data Lake and a Splunk index.

   • For a raw table landing definition, create and activate an Ingest Processor pipeline with the same matching condition and a Splunk index destination for the preserved index. Use Ingest Processor, not Edge Processor. See Preserve a Splunk index path when using a data landing definition.

3. Record the values and behavior that you will validate.

   Matching events

   Record the source, sourcetype, host, time range, partition, or filter values that identify events intended to land in Machine Data Lake.

   Target raw table

   Record the raw table name and the resource name used in configuration, such as ~.federated.<raw_table_name>.

   Preserved Splunk index path

   If matching events must remain available in a Splunk index, record the destination index and the procedure that preserves that route.

   Non-matching events

   Record where events that do not match the landing definition or test condition should go.

   Processing and rollback

   Record any supported filters, masking, Heavy Forwarder processing, Ingest Processor actions, and the previous configuration that you can restore if validation does not match the plan.

4. Configure the selected route and send a uniquely identifiable test event.

   Use a value that is not already present in production data, such as mdl_test_id=<unique_id>. The direct-routing and dual-routing child tasks include copy-ready examples for sending or generating the test event.

5. Validate the test event in every expected destination.

   • Verify the uniquely identifiable event in the Machine Data Lake raw table by using a narrow time range and the known source, sourcetype, or host values.

   • If you configured dual routing, verify the same unique event in the preserved Splunk index.

   • If the route uses a raw table landing definition, verify that representative non-matching events do not land in the raw table and still follow the intended route.

   Use raw search, index search, or preview for immediate validation. Catalog metadata, event ranges, field summaries, and dataset statistics can refresh later.