Model Runtime in Splunk Security Assistant

Splunk Security Assistant provides the option to use the large language models (LLMs) hosted in Splunk Cloud Platform or models hosted in Azure OpenAI. When you use the Model Runtime feature, letting Splunk determine the best model to use, Splunk Security Assistant determines when to use a Splunk platform hosted LLM, and when to use a third-party LLM, based on your prompt. Third-party LLMs can provide better response quality through the assistant, depending on factors such as use case and cost. ES 8.6 or higher uses Model Runtime by default. Administrators can turn off this functionality at any time from the Settings page.

Using the Model Runtime feature

When you choose to let Splunk determine the best model to use, Splunk AI Assistant can leverage an external large language models (LLM) hosted in Azure OpenAI. This LLM generates the response provided by the app when deemed necessary, and can improve the response quality.

Splunk Security Assistant leverages the additional options from the LLM based on the intent and complexity of the request. The external LLM endpoint is secure but is outside the Splunk platform data boundary. The search prompt is sent to the third-party LLM and is governed by the third-party LLM provider's data handling policy.

The Model Runtime feature includes enterprise-grade compliance and regional data boundaries. Opting in causes no disruption to Splunk Security Assistant services or responsiveness.

When you opt-in, search responses are tagged with the source as being either internal, using the Splunk platform, or external, using the third-party LLM. Administrators can view these audit log tags as needed.

Model Runtime feature availability and region standard

See the following table for each supported region for Model Runtime, when the feature became available in that region, and the region standard. Region standard shows if your requests to the app might be processed outside the selected region.

Region standards are defined as follows:

• Data zone standard: App requests can route to any region within the same zone. Provides zone-level routing only.

• Global standard App requests can route anywhere in the world. Does not provide zone-level or country-level routing guarantees.

Supported regions

You can only use the Model Runtime feature if you are running the assistant in a supported region. Model Runtime is supported for Splunk AI Assistant users in the following regions:

• AWS - Canada Central

• AWS - AP Mumbai

• AWS - AP Seoul

• AWS - AP Singapore

• AWS - AP Sydney

• AWS - AP Tokyo

• AWS - EU London

• AWS - EU Frankfurt

• AWS - EU Dublin

• AWS - EU Milan

• AWS - EU Paris

• AWS - US West Oregon

• AWS - US East Virginia

• AWS - SA São Paulo

• Azure - East US (Virginia)

• Azure - UK South (London)

• Azure - West US (California)

• Azure - Japan East (Tokyo)