Connections tab in the AI Toolkit

Use the Connections tab to configure connections for both LLMs and containers. The LLM connection type lets you select the appropriate LLM provider, while the Container connection type lets you choose between Docker and Kubernetes container types, primarily for Splunk App for Data Science and Deep Learning (DSDL) use cases.
Note: In versions lower than version 5.7.0 of the toolkit, this tab is labeled as Connection Management and only includes the option to connect to an LLM.

Add an LLM connection

Log in to the AI Toolkit, navigate to the Connections tab, and choose Add a Connection as shown in the following image:

This image shows the Connections tab in the AI Toolkit. A icon on this tab labeled as Add a Connection is highlighted.
CAUTION: By using the LLM connection feature you acknowledge and agree that data from searches using the ai command is processed by an external LLM service provider.

LLM connection permission

To select LLM as the Connection Type, you must have the following capability from the mltk admin role:

  • list_ai_commander_config

Add a Connection steps

Complete the following steps

  1. Enter a name into the Connection name field.

  2. From the Connection Type drop-down menu, choose LLM.

  3. Make a selection from the Provider drop-down menu. See Supported LLM providers in the next section.

  4. Complete the Input connection details fields. What fields display depends on which LLM provider is selected from the Provider menu.

  5. (Optional) Use the Test Connection option from the top right and adjust field values as needed.

  6. Save the connection.

After the connection has been configured, you can use the LLM with the ai command. For more information see About the ai command.

Note: The ai command does not inspect the input to the LLM. Use discretion to determine if the data you send to the LLM is suitable and appropriate.
The following image shows an example of the LLM connection set up page:

This image shows the set up a new connection page where LLM is the chosen connection type.

Supported LLM providers

You can use the ai command with the following LLM providers:

Note: Splunk Hosted Models is a new option provided in AI Toolkit version 5.7.0 and higher and offers access to hosted language models to integrate LLM capabilities into your workflow. Available options are OpenAI GPT-OSS 120B, OpenAI GPT-OSS 20B, and Llama-3.1-FoundationAI-SecurityLLM-base-1.1-8B.
  • Splunk Hosted Models
  • OpenAI

  • Anthropic

  • AzureOpenAI
  • Groq
  • Gemini
  • Bedrock
  • Ollama

You can use the provider= or model= parameters in your ai command search to switch between these providers.

Add a Container connection

When following the steps to add a new connection, the Connection Type drop-down menu provides the options of LLM or Container. When you select Container you can create a Kubernetes or Docker external runtime connection, or enable HPA.

Log in to the AI Toolkit, navigate to the Connections tab, and choose Add a Connection as shown in the following image:

This image shows the Connections tab in the AI Toolkit. A icon on this tab labeled as Add a Connection is highlighted.

Container connection permissions

To select Container as the Connection Type, you must have the following capability from the mltk admin role:

  • list_ai_commander_config

To select Container as the Connection Type, you must have the following capabilities from the dsdl_admin role:
Note: These capabilities allow you to define and configure external run times. For example creating a Kubernetes cluster definition in the AI Toolkit.

  • list_container_connections

  • setup_container_configuration

  • enable_hpa

To run the fit and apply commands for a specific AI Toolkit container you must have the following capabilities from the dsdl_admin role:

  • fit_mltkcontainer

  • apply_mltkcontainer

Add a Connection steps

Complete the following steps:

  1. Enter a name into the Connection name field.

  2. From the Connection Type drop-down menu, choose Container.

  3. Make a selection from the Provider drop-down menu. Kubernetes and Docker are both supported.

  4. Complete the Input connection details fields. What fields display depends on which container provider is selected from the Provider menu.

  5. (Optional) Use the Test Connection option from the top right and adjust field values as needed.

  6. Save the connection.

Note: By using the container connection feature you acknowledge and agree that data from searches initiated from this deployment will be stored and processed in an external container service provider.

The following image show an example of the Container connection set up page:

This image shows the set up a new connection page where Container is the chosen connection type and Kubernetes as the provider.