Share data in Splunk AI Assistant for SPL

When you interact with Splunk AI Assistant for SPL, Splunk may use your chat history (including inputs and outputs), context data collected from your environment as noted in this section and updated from time to time, and in-product feedback you give to develop and improve the assistant, including for Splunk research and development which may include training our models.

If you do not want to share data to be used for these purposes, you may toggle this collection off in Settings tab of the app.

How to opt in or out of sharing data for research and development

Data sharing is turned on by default. You can turn data sharing off from within Splunk AI Assistant for SPL on the Settings tab of the app. Deselect the box next to "Share AI usage data with Splunk" as shown in the following image:

This image shows the Settings tab of Splunk AI Assistant for SPL. A tick-box labeled as Share AI usage data with Splunk is highlighted. From this page in the app you can choose to share or not share some data with Splunk.

What data is collected

Splunk AI Assistant for SPL collects different context data depending on if you opt-in to share data and opt-in to use the Personalization feature.

Share data

In addition to your chat history, including inputs and outputs, and in-product feedback, Splunk AI Assistant for SPL collects the following context data:

Component Description Example
app.session.copy_spl_clicked Data collected when SPL generated using the app is copied with the "Copy" button. 
app: splunk_instrumentation

   component: app.session.copy_spl_clicked

   data: { [-]
     app: Splunk_AI_Assistant_Cloud
     page: dashboard
     source: SAIA UI Telemetry
     spl: index=_internal sourcetype=splunkd log_level=ERROR| timechart count| rename _time as Time, count as Count
}
app.Splunk_AI_Assistant Information including type, tenant, query, enabled_features, and request_id. 
{
   'type': 'inference_spl_generation',
    'tenant': 'saia-stg-custom',
     'query': ' SAIA has expert knowledge of the Splunk platform and Splunk...',
     'enabled_features': "['customization']", 
'request_id' : c88bbad8-92ab-4851-ac5f-b417b984f53c
}
app.Splunk_AI_Assistant Information including tenant, and type. 
{
     'type': 'customization_opt_in',
       'tenant': 'saia-stg-custom'
}
app.Splunk_AI_Assistant.splgen Collects the chat_id. 
{
....
'chat_id': 4
}
app.Splunk_AI_Assistant.splgen.feedback Information including enabled_features, feedback_id, and query. 
{   
    enabled_features : ['customization']
    feedback_id : '4e618319-2276-4ae7-9436-ab2713735629'
       query : 'List available indices'
}
app.Splunk_AI_Assistant_Cloud.splgen Logging from Splunk AI Assistant for SPL Splunk app REST handlers. 
2024-05-27 16:26:25 UTC, Level=INFO, Pid=1063271, Logger=ChatHistoryHandler, File=chat_history_handler.py, Line=43, UUID="34547aed-648c-4d3f-b2ce-f1ce066a57ad", message="Handling chat history request"
app.Splunk_AI_Assistant_Cloud.splgen Generation time. End to end (e2e) time from request start to end. 
2024-05-24 18:05:50 UTC, Level=INFO, Pid=2248783, Logger=AsyncHttpJobs, File=jobs.py, Line=87, UUID="4475f233-2559-42ee-b7ff-c2891ae0d549", apply_time="2.16974", user="haydn"
app.Splunk_AI_Assistant_Cloud.splgen.openinsearch When the user clicks on the "Open in Search" button for some generated SPL. 
{ 
"data": {
"_time": 1688763330,
"_sourcetype": "splgen_feedback",
"session_id": "1dd4af3e-a567-4d68-a491-75964913d868",
"spl": "'| rest splunk_server=local /services/cluster/master/peers | stats sum(bucket_count) by label | rename label as peer'",
"user": "<hashed username>",
"_kv": 1,
"_serial": 0 }
}
app.Splunk_AI_Assistant_Cloud.splgen.usage Feedback submitted by users with thumbs up/thumbs down/additional details UI in app. 
{ 
"data": {
"_time": 1688763330,
"response": "'Concise Summary:\nThe query retrieves the total number of buckets per peer in a Splunk cluster.\nDetailed Explanation:\n- `| rest splunk_server=local /services/cluster/master/peers`: This part of the query uses the REST command to access the local Splunk cluster master'",
"_sourcetype": "splgen_feedback",
"session_id": "1dd4af3e-a567-4d68-a491-75964913d868",
"query": "'| rest splunk_server=local /services/cluster/master/peers | stats sum(bucket_count) by label | rename label as peer'",
"correct": "true",
"_kv": 1,
"_serial": 0 }
}
inference_spl_generation

inference_spl_explanation

Natural language prompt entered by the user in user_prompt field and intermediate rag/metadata responses retrieved from the large language models (LLMs). 
 {
'user_prompt' : "show storage freespace in winhostmon",
'retrieved_rag': ```search 'search index=windows sourcetype=WinHostMon Type=Disk | table host, Name, DriveType, TotalSpaceGB, FreeSpaceGB, FreeSpacePct | sort FreeSpacePct'```,
'retrieved_personalization_metadata': ['component', 'datetime', 'log_level', 'data.total_size', 'data.name', 'dns_alt_name', 'sh_label', 'data.total_bucket_count', 'data.bucket_dirs.cold.capacity', 'data.bucket_dirs.home.capacity'],
'generated_response': ``` index=windows sourcetype=WinHostMon Type=Disk | stats sum(FreeSpaceKB) as total_free_space by Name | eval total_free_space_GB = round(total_free_space / 1024 / 1024, 2) | table Name, total_free_space_GB ```
}
saia-tenant-id Hashed name of the tenant or stack ID. 
{
   .....
    saia-tenant-id: 1b366eb2-3dfa-520e-b353-8178af77cfbd

   sourcetype: saia_api_event
}
stackID

userID

chat_id
app_version

Information collected from the StackID, UserID, ChatID, and App Version fields. 
{
stackID=CLOUD-7e42604c501e415b0b72b841bd788e84db49ea089713d9a5afe2a17d74e9b7a9,
userID=677ee9314a5407cfdb0a224f,
chat_id=0,
app_version="1.0.6",
}
job_id

user_key

user
chat_id

Information collected from the JobID, UserKey, User, and ChatID fields. 
....
request_id: 
job_id=5637081e-ab41-432d-bce9-9f76c61c9b1c
user_key=677ee9314a5407cfdb0a224f
chat_id=0
user=2340314992997373707
}
input_word_count

input_char_count

output_word_count
output_char_count

Total numbers of the word and character counts for input and output responses. 
{
input_char_count: 115

input_word_count: 20

output_char_count: 1896

output_word_count: 236
}
source_app_id SourceAppID information. 
source_app_id: Splunk_AI_Assistant_Cloud_Custom
num_distinct_clusters

avg_clusters_per_srctype

avg_fields_per_cluster min_fields_per_cluster max_fields_per_cluster

Information collected on distinct clusters formed for each tenant, average number of clusters formed per sourcetype, average number of field lists collected per cluster, minimum number of fields per cluster, and maximum number of fields per cluster. 
{
num_distinct_clusters: 11
avg_clusters_per_srctype: 2
avg_fields_per_cluster: 4.5
min_fields_per_cluster: 1
max_fields_per_cluster: 139
}

Personalization data

Personalization is turned off by default. You can turn data sharing on or back off from within Splunk AI Assistant for SPL on the Settings tab of the app. Deselect the box next to "Personalize results" as shown in the following image:

This image shows the Settings tab of Splunk AI Assistant for SPL. The toggle for Personalize results is highlighted.

The following context data is collected if you opt-in to use Personalization.

This data is collected using 2 saved searches bundled with the assistant. These searches are only enabled if you opt-in for Personalization:

  • Splunk AI Assistant for SPL - Field Summary
  • Splunk AI Assistant for SPL - Search Logs

Collected data is stored in the vector DB, and a cleanup job runs weekly to delete this information if you decide to opt-out of Personalization at a later date.

Component Description Example
app.Splunk_AI_Assistant.index_metadata Sourcetype metadata 
{
"tenant": "caeinternal1",
"index_metadata": "[{ 'max': '2846', 'min': '0', 'mean': '2.054869684499314', 'count': '3645', 'field': 'duration_command_search_rawdata', 'index': 'main', 'sourcetype':'audittrail', 'stdev': '51.19505709576045', 'is_exact': '1', 'distinct_count': '33', 'numeric_count': '3645', 'is_numeric': True}]"
}
app.Splunk_AI_Assistant.previous_searches Previous searches 
{
              "tenant": "saia-play-custom",
               "searches": [
                  {
                      "user": "admin",
                      "spl": "| search index=\"_internal\" sourcetype=\"splunk_ai_assistant-3\" | fieldsummary | eval index=\"_internal\", sourcetype=\"splunk_ai_assistant-3",
                       "count": 1,
                        "roles": ["admin" , "mltk_model_admin"]
                    },
                  {
                      "user": "admin",
                       "spl": "| search index=\"_introspection\" sourcetype=\"splunk_telemetry\" | fieldsummary | eval index=\"_introspection\", sourcetype=\"splunk_telemetry\"",
                     "count": 1,
                      "roles": ["admin" , "power_user", "mltk_model_admin"]
                 }
           ]
}
num_indexes

num_distinct_indexes

num_sourcetypes
num_distinct_sourcetypes
average_sourcetype_per_index
num_spls
num_distinct_spls
num_users
num_distinct_users
average_spls_per_user

VectorDB metrics for all the tenants who opted for the personalization feature. 
{
average_spls_per_user: 1

num_distinct_spls: 11

num_distinct_users: 2

num_spls: 11

num_users: 11

}
........

{
average_sourcetype_per_index: 6.625

num_distinct_indexes: 8

num_distinct_sourcetypes: 49

num_indexes: 53

 num_sourcetypes: 53

}

Data retention

Data shared as outlined in this section is retained as set forth in the Splunk Data Retention Policy.

Chat data is stored in the KVStore on the customer's stack. If you choose to delete a chat, that chat data is deleted from your local KVStore collection.

Note: If you opt-in for the personalization feature preview available with versions 1.0.5,1.0.6, and 1.1.0, that collected data is stored in the vector database. If you opt-out of this personalization preview at a later date, a cleanup job runs weekly to delete any collected data.