Onboard your first data source

Validate one supported data source by landing a uniquely identifiable test event in a raw table, checking any required Splunk index route, and choosing the next supported action.

  • You completed the required environment, service-account, permission, and routing checks. See Machine Data Lake prerequisites.

  • You prepared a supported input path, such as HTTP Event Collector (HEC), Universal Forwarder, or Heavy Forwarder. See Prepare a data source for Machine Data Lake.

  • You know whether matching events land only in Machine Data Lake or must also continue to an existing Splunk index path during onboarding, migration, validation, or rollback planning. See Choose a Machine Data Lake data strategy.

  • You have permission to create or edit the raw table, preview the landing definition, search the raw table, and search any preserved Splunk index path.

Use this tutorial for the first source that you onboard into Machine Data Lake. The goal is to prove the complete data path with one easy-to-find event before you rely on the source for production retention, search, promotion, sharing, or downstream analytics.

Onboarding is successful when you can answer these questions:

  • Did the matching test event land in the raw table?

  • Did any existing Splunk index path still receive the event where required?

  • Did representative non-matching data stay out of the raw table?

  • Is raw search enough for the use case, or does the data need promotion?

Note: Use search, preview, or event inspection for immediate validation. Don't rely only on the Catalog metadata, dataset statistics, field values, or field summaries for recent events because those details refresh on a background schedule.
  1. Create a unique test value for this onboarding run.

    Use a value that is not already present in production data, such as mdl_test_id=<unique_test_id>. Include the value in the sample data used for landing preview and in the real event that you send through the selected input path.

  2. Confirm the source values that identify matching events.

    Record the source, sourcetype, host, and time range values for the events that must land in Machine Data Lake. Also record one representative event that must not match the landing definition.

  3. Confirm the routing behavior for matching and non-matching events.

    For matching events, choose whether they land only in Machine Data Lake or also continue to an existing Splunk index path where supported. For non-matching events, confirm the default or preserved destination. Keep the previous configuration available so that you can restore it if validation does not match the plan.

  4. Create or open the raw table for the source.

    If the source doesn't have a raw table yet, create one and define dataset access, retention, promotion limits, and landing behavior.

    If the source already has a raw table, open the dataset and edit the landing definition.

    See Create a Machine Data Lake raw table and Inspect and edit raw tables

  5. Preview the landing definition with matching and non-matching sample data.

    Include the unique test value in the matching sample event. Confirm that the matching sample event is included, that the non-matching sample event is excluded from the raw table, and that filters, masking, custom command functions, or other actions produce the expected output.

  6. Save the raw table or landing-definition change only after preview matches the plan.

    If preview is empty or unexpected, revise the sample data, partitions, source values, filters, masking, actions, or routing configuration before you save.

  7. Send the uniquely identifiable test event through the selected input path.

    Replace the placeholders in these examples with values from your environment.

    Input path Example test event pattern
    HEC
    JSON 
    curl https://<hec_endpoint>/services/collector/event -H "Authorization: Splunk <hec_token>" -d '{"event":"mdl_test_id=<unique_test_id> action=mdl_onboarding_test","sourcetype":"<test_sourcetype>","source":"<test_source>","host":"<test_host>"}'
    Universal Forwarder

    Write one line that contains mdl_test_id=<unique_test_id> to a monitored test file or source that the Universal Forwarder sends to the Machine Data Lake-enabled Splunk Cloud Platform environment.
    Heavy Forwarder

    Send one event that contains mdl_test_id=<unique_test_id> through the Heavy Forwarder input, parsing, filtering, and forwarding path that you plan to use for production data.
  8. Verify that the test event landed in the raw table.

    Open the raw table from the Catalog and search or inspect the data. Use a narrow time range that includes when you sent the test event, and constrain the search with supported fields such as source, sourcetype, and host. Inspect the returned raw event for the unique test value.

    See Find Machine Data Lake datasets in the Catalog and Search Machine Data Lake data from the Catalog

    CODE 
    sourcetype="<test_sourcetype>" host="<test_host>" mdl_test_id="<unique_test_id>" earliest=-30m latest=now
  9. If matching events must continue to an existing Splunk index path, verify the same test event in that path.

    Search the existing Splunk index for the same unique test value and time range. Confirm that preserving the existing path didn't prevent the event from landing in Machine Data Lake.

    CODE 
    index=<preserved_index> mdl_test_id="<unique_test_id>" earliest=-30m latest=now
  10. Verify the expected behavior for non-matching data and delayed metadata.

    Confirm that representative non-matching events don't appear in the raw table and follow the intended default or preserved destination. If the event is searchable but the Catalog field values, event range, or dataset statistics don't show the update yet, allow for metadata refresh latency before treating the issue as a landing failure.

    See Machine Data Lake limits and caveats and Troubleshoot Machine Data Lake

  11. Choose the next action based on the validation result.
    • If the test event doesn't land as expected, restore the previous input, routing, or landing configuration. Revise the source values, partitions, filters, masking, actions, or permissions, and then send another uniquely identifiable test event.
    • If the use case requires indexed search, dashboards, alerts, lookups, or threat-intelligence workflows, promote the required data to a Splunk index. If the use case requires a defined schema for structured analytics, promote data to an analytics table. Use Open Sharing separately when a supported external client needs access; Open Sharing is not a promotion destination.If raw search is enough for validation and occasional inspection, keep the data in the raw table and do not create a promotion.

The first data source is validated. You confirmed that the matching test event lands in Machine Data Lake, that any preserved Splunk index path still receives the event where required, that representative non-matching data follows the intended route, and that the source has the right next action.

In this validation example, a web access log source sends events by HEC with sourcetype="access_combined", source="/var/log/nginx/access.log", and host="web-01". The landing definition selects that sourcetype and host. The test event includes mdl_test_id=mdl-onboard-20260729-01.

Preview should include the web access log test event and exclude a representative event with a different sourcetype, such as an operating system audit log. After you send the real event, raw search should return the web access log event for the narrow time range. If the source also keeps an existing Splunk index path, the index search should return the same test ID. The Catalog field summaries might show the new value later.

After validation succeeds, use the Catalog to find the raw table, inspect dataset details, search raw data, create a promotion if needed, or manage the dataset.

See: