Query and access Machine Data Lake data

Choose the supported query or access path for a raw table, Splunk index, analytics table, Open Sharing client, or external Spark workflow.

Choose the supported query or access path for a raw table, Splunk index, analytics table, Delta Sharing client, or external Spark workflow.

For general guidance about starting a search from the Catalog and continuing in the Search & Reporting app, see Search, transform, and analyze your data. Use this topic to understand the Machine Data Lake-specific behavior for each dataset type.

Raw search

Raw search is useful for validating that landed data exists and for narrowing the next action. Raw tables are storage-optimized, not search-optimized, so raw search might not provide the performance or field behavior required for repeated operational searches.

The search experience can warn that raw search can be costly before it opens raw search. Proceed when raw search is the right validation path and, where supported, choose not to show the warning again.

Raw search supports the Machine Data Lake metadata fields documented for the raw table search surface, including source, sourcetype, host, and time. Do not assume that derived or implicitly extracted fields are available unless the raw-search documentation lists them. When the use case requires indexed search, dashboards, alerts, lookups, or correlation, promote the required data to a Splunk index. When it requires a defined schema for structured analytics, promote the required data to an analytics table. Do not search the full raw table by default.

Analytics search

Use analytics search when a static promotion has created an analytics table for a specific historical time range. An analytics table uses the exact schema defined during promotion, including fields extracted from raw data or shaped with custom SPL2.

Choose this path when you know which fields you need. Select only those fields during promotion so the table stays focused and easier to search. Because analytics tables use a columnar format, you can scan selected fields across large data volumes more efficiently than you can scan full raw events.

Use analytics tables for work such as threat hunting, compliance reports, audit review, trend analysis, large-scale aggregation, and machine learning training. For example, you can promote one month of process logs with only the fields you need for command-line analysis, or promote audit events with the fields required for a compliance report.

If Open Sharing is supported for your environment and dataset type, you can share an analytics table with authorized tools or workflows outside Splunk. Use Open Sharing when a business intelligence tool, notebook, or machine learning pipeline needs read-only access to the promoted data without a manual export.

Open Sharing uses the Delta Sharing protocol for the current Machine Data Lake sharing workflow. Delta Sharing is an open REST protocol for sharing access to large cloud datasets. In Open Sharing, the external sharing profile gives a supported client the sharing endpoint and bearer token. The client requests table metadata from the sharing service and reads the authorized data files.

To share an analytics table with Open Sharing, complete the following steps:

  1. Confirm that Open Sharing is supported for your Splunk Cloud Platform environment and dataset type. Verify that your role includes the share_datasets_external capability and the required dataset access. For permission guidance, see Access control, roles, and capabilities.

  2. Choose the analytics table to share. If the external workflow needs data from a raw table, promote the required time range, filters, and fields to an analytics table first.

  3. Open the analytics table details from the Catalog, then select the Manage tab.

  4. Set the profile expiration or access duration, then select Enable sharing. After sharing is enabled, use Download external sharing profile to download the JSON profile for the client. The profile includes shareCredentialsVersion, endpoint, bearerToken, and expirationTime.

  5. Configure a supported Delta Sharing client with the sharing profile. Depending on validated support for your environment, supported clients include Python Delta Sharing clients, pandas, and Power BI. For more Delta Sharing integrations, see Delta Sharing integrations.

  6. Read the latest snapshot of the shared table from the external client. For the current Machine Data Lake sharing workflow, do not depend on time travel (querying earlier table snapshots by timestamp or version), Change Data Feed, Delta table versions, or commit history unless Splunk documents support for those features. For Delta Sharing read examples, see Read Delta Sharing Tables.

  7. When the external workflow no longer needs access, let the profile expire or select Revoke sharing. Expiration or revocation stops access through that profile, but it does not delete the analytics table or promotion job.

Important:

Treat sharing profiles, bearer tokens, profile secrets, and presigned URLs as sensitive information. Do not include these values in support cases, troubleshooting notes, chat messages, or shared logs.

Splunk records Open Sharing profile lifecycle and access events in audit data where Open Sharing is available.

Do not use this workflow for federated access. With Open Sharing, an external client reads a Machine Data Lake analytics table through a sharing profile. With federated access, Splunk queries data that remains in another supported system.

Before you create the promotion, confirm the time range, filters, and fields that your work requires. Select only the fields that support your analysis.

This search path depends on modules for Search Processing Language version 2 (SPL2) and on data orchestration services. Search availability depends on those services being available and healthy.

Splunk search

Use Splunk search for a promoted Splunk index. Open target. Open the supported Splunk search surface from the Catalog or the promotion job details, and use the search syntax documented for that surface.

Open Sharing and external access

Use Open Sharing when a supported external client needs authorized, read-only access to a raw table or analytics table. Open Sharing uses Delta Sharing profiles that can expire or be revoked. Supported clients, including Spark-based workflows, can read shared tables without a manual export. Federated datasets are different: the data remains in another supported system and is queried in place; it is not a Machine Data Lake raw table or promotion target.