Route HEC events directly to Machine Data Lake

Create an HTTP Event Collector token that routes events to a Machine Data Lake raw table and send a test event.

  • You have a Machine Data Lake raw table and know its resource name, such as ~.federated.<raw_table_name>.

  • You have permission to create or edit HTTP Event Collector (HEC) tokens in Splunk Cloud Platform.

  • The HEC client can use a token that targets the raw table.

Use this route when a HEC client can use a token that targets the Machine Data Lake raw table. The HEC client can omit the index field when the token default identifies the raw table, or include the raw table resource name in the payload when that destination is allowed for the token.

  1. Start creating a HEC token in Splunk Web.

    In Splunk Cloud Platform, select Settings > Add Data, select Monitor, and then select HTTP Event Collector.

  2. In Name, enter a token name that clearly identifies the raw table, such as cisco_asa_mdl. Optionally enter a source name override, description, or other HEC settings required by your organization.
  3. On the input settings or index selection step, select the raw table whose resource name is ~.federated.<raw_table_name>.

    Set that raw table as the default destination or default index, and include it in the allowed destinations or allowed indexes. If the UI shows the raw table display name instead of the resource name, confirm that the selected dataset is the target Machine Data Lake raw table before you continue.

  4. Review the token settings. If the settings identify the correct raw table, submit the token and wait for Splunk Cloud Platform to deploy it.
  5. Copy the token value after the token is created and deployed.

    If deployment status is available, wait until the deployment is complete before you use the token in a HEC client.

  6. Send a uniquely identifiable test event with the Machine Data Lake HEC token.
    JSON 
    curl "https://http-inputs-<stack>.splunkcloud.com/services/collector/event" -H "Authorization: Splunk <mdl-token>" -H "Content-Type: application/json" -d '{"event":"mdl_test_id=<unique_id> action=direct_hec_test","sourcetype":"cisco:asa","source":"hec:firewall","host":"asa-01"}'

    This example omits the index field because the HEC token default identifies the raw table.

  7. If the HEC token allows the raw table destination and you need the payload to identify it explicitly, include the index field.
    JSON 
    curl "https://http-inputs-<stack>.splunkcloud.com/services/collector/event" -H "Authorization: Splunk <mdl-token>" -H "Content-Type: application/json" -d '{"index":"~.federated.<raw_table_name>","event":"mdl_test_id=<unique_id> action=direct_hec_test","sourcetype":"cisco:asa","source":"hec:firewall","host":"asa-01"}'
  8. Verify the test event in the raw table.

    Open the raw table from the Catalog and search for the unique test value by using a narrow time range.

    CODE 
    mdl_test_id="<unique_id>" sourcetype="cisco:asa" source="hec:firewall" host="asa-01"

The HEC request succeeds and the event lands in the Machine Data Lake raw table identified by the token default or by the allowed index value. No event is written to a Splunk index unless you configure a separate supported route.

For more information, see Set up and use HTTP Event Collector in Splunk Web and Format events for HTTP Event Collector.