Connections tab in the AI Toolkit

Use the Connections tab to configure connections for both LLMs and containers. The LLM connection type lets you select the appropriate LLM provider, while the Container connection type lets you choose between Docker and Kubernetes container types, primarily for Splunk App for Data Science and Deep Learning (DSDL) use cases.
Note: In versions lower than version 5.7.0 of the toolkit, this tab is labeled as Connection Management and only includes the option to connect to an LLM.
Note: If you experience issues with drop-down menus on the Connections tab populating, try using Chrome as your browser.

Add an LLM connection

Log in to the AI Toolkit, navigate to the Connections tab, and choose Add a Connection as shown in the following image:

This image shows the Connections tab in the AI Toolkit. A icon on this tab labeled as Add a Connection is highlighted.
CAUTION: By using the LLM connection feature you acknowledge and agree that data from searches using the ai command is processed by an external LLM service provider.

LLM connection permissions

To select LLM as the Connection Type, you must have the following capability from the mltk admin role:

  • list_ai_commander_config

To select Splunk Hosted Models from the Provider drop-down menu, you must have the following capability:

  • list_tokens_scs

Add a Connection steps

Complete the following steps

  1. Enter a name into the Connection name field.

  2. From the Connection Type drop-down menu, choose LLM.

  3. Make a selection from the Provider drop-down menu. See Supported LLM providers in the next section.

  4. Complete the Input connection details fields. What fields display depends on which LLM provider is selected from the Provider menu.

  5. (Optional) Use the Test Connection option from the top right and adjust field values as needed.

  6. Save the connection.

After the connection has been configured, you can use the LLM with the ai command. For more information see About the ai command.

Note: The ai command does not inspect the input to the LLM. Use discretion to determine if the data you send to the LLM is suitable and appropriate.
The following image shows an example of the LLM connection set up page:

This image shows the set up a new connection page where LLM is the chosen connection type.

Supported LLM providers

You can use the ai command with the following LLM providers:

Note: The Splunk Hosted Models provider option is available in AI Toolkit version 5.7.0 and higher and offers access to hosted language models to integrate LLM capabilities into your workflow. Available options are OpenAI GPT-OSS 120B, OpenAI GPT-OSS 20B, and Llama-3.1-FoundationAI-SecurityLLM-base-1.1-8B. You must have the list_tokens_scs capability to see this option.
  • Splunk Hosted Models
  • OpenAI

  • Anthropic

  • AzureOpenAI
  • Groq
  • Gemini
  • Bedrock
  • Ollama

You can use the provider= or model= parameters in your ai command search to switch between these providers.

Bedrock configuration steps

You must set up an AWS IAM Role and IAM User to integrate with Amazon Bedrock, and configure these credentials in the Connection Management page.

Note: Without taking these steps, when you select Bedrock from the Provider drop-down menu, you will not see any selectable options from the Select Model drop-down menu.

Compete the following steps:

  1. Create an IAM Role with AmazonBedrockFullAccess Policy:

    1. Sign into your AWS account and navigate to the IAM Console - Roles page. See https://console.aws.amazon.com/iam/home#/roles

    2. Select Create Role.

    3. Select Trusted Entity Type as AWS account.

    4. Choose Another AWS account or your own account, as applicable.

    5. In the Permissions policies section, search and select AmazonBedrockFullAccess.

    6. Complete the role creation steps and note down the Role ARN as shown in the following example: arn:aws:iam::<account_id>:role/mltk-bedrock-fullaccess-role

      The following image shows an example view of a completed create roles page:

      This image shows an example of an AWS account page and role information.

  2. Create IAM User with Assume Role Permissions:

    1. In your AWS account navigate to the IAM Console - Users page. See https://console.aws.amazon.com/iam/home#/users

    2. Select Add users.

    3. Enable Access key – Programmatic access.

    4. In the Permissions step, choose Attach policies directly and select no policies.

    5. Complete the user creation. Make note of the Access Key ID and Secret Access Key.

    6. Edit the user to attach the following inline policy to allow assume-role access:

      JSON 
      {    
"Version": "2012-10-17",    
"Statement": [        
{            
"Sid": "Statement1",            
"Effect": "Allow",            
"Action": "sts:AssumeRole",            
"Resource": "arn:aws:iam::<account_id>:role/mltk-bedrock-fullaccess-role"        
}    
]
}

    7. Replace <account_id> with your actual AWS account ID. The following image shows an example view of the completed permissions page:

      This image shows an example of an AWS account page and permissions information.

    8. Add the newly created user to the Trust relationships and the role created in step 1.

Add a Container connection

When following the steps to add a new connection, the Connection Type drop-down menu provides the options of LLM or Container. When you select Container you can create a Kubernetes or Docker external runtime connection, or enable HPA.

Log in to the AI Toolkit, navigate to the Connections tab, and choose Add a Connection as shown in the following image:

This image shows the Connections tab in the AI Toolkit. A icon on this tab labeled as Add a Connection is highlighted.

Container connection permissions

To select Container as the Connection Type, you must have the following capability from the mltk admin role:

  • list_ai_commander_config

To select Container as the Connection Type, you must have the following capabilities from the dsdl_admin role:
Note: These capabilities allow you to define and configure external run times. For example creating a Kubernetes cluster definition in the AI Toolkit.

  • list_container_connections

  • setup_container_configuration

  • enable_hpa

To run the fit and apply commands for a specific AI Toolkit container you must have the following capabilities from the dsdl_admin role:

  • fit_mltkcontainer

  • apply_mltkcontainer

Add a Connection steps

Complete the following steps:

  1. Enter a name into the Connection name field.

  2. From the Connection Type drop-down menu, choose Container.

  3. Make a selection from the Provider drop-down menu. Kubernetes and Docker are both supported.

  4. Complete the Input connection details fields. What fields display depends on which container provider is selected from the Provider menu.

  5. (Optional) Use the Test Connection option from the top right and adjust field values as needed.

  6. Save the connection.

Note: By using the container connection feature you acknowledge and agree that data from searches initiated from this deployment will be stored and processed in an external container service provider.

The following image show an example of the Container connection set up page:

This image shows the set up a new connection page where Container is the chosen connection type and Kubernetes as the provider.