Feature preview: Optimization

Versions 1.3.0 and 1.3.1 of Splunk AI Assistant for SPL offers a preview of a new feature - optimization. Optimization can improve existing SPL searches or generate better searches when used with the Write SPL feature.

Note: Participation in this preview feature is optional and can be turned on or off at any time.

Use optimization to run faster searches and reduce costs with similar results. Through the Write SPL tab you can optimize existing searches, including saved searches, or compose new SPL searches that are then optimized.

Requirements

The optimization feature uses metadata from your Splunk deployment to provide better results from Splunk AI Assistant for SPL. Opting into personalization is a requirement to using optimization. See Configure Personalization settings

In order to participate in the preview, you must also share your AI Service Data so that we can gather the necessary feedback to improve this feature before general availability. See Configure share data settings.

Participating in the preview

If you are a user with administrator privileges, when you install version 1.3.1 or upgrade to version 1.3.1, you will see the following modal window and the option to opt-in:

This image shows the modal window that will display when you open version 1.3.0 of the assistant. Administrator users can choose to turn on the SPL Optimization preview on this window, or opt in or out later through the Settings page.

Only users with administrator privileges can opt-in or opt-out of this preview. You can opt-in or out at any time and the change takes effect immediately.

Whether the optimization preview is turned off or on, that setting applies at the app level, across all users, and not at the individual user level.

Note: When you opt-in for optimization the name of the Write SPL tab updates to be the Write & optimize SPL tab.

If you want to opt-in or out of this preview feature at a later date than the app installation or upgrade, navigate to the Settings tab of the assistant. Select or deselect the Optimize SPL option, as shown in the following image:

This image shows the information available on the Settings tab of the assistant. The toggle for SPL Optimization is highlighted.

Note: Users without administrator privileges see the optimization information and the setting chosen, but cannot change this setting.

Optimization strategies

The following optimization strategies are leveraged in this feature preview:

Choosing the right index
Using tstat
Eliminating unnecessary wildcards

Note: More strategies will be added during the preview stage.

Optimization examples

To optimize an existing search, you can paste that existing search in the Write & optimize SPL tab. Alternately, you can ask the assistant to perform optimization using the format of Optimize this search <original_search_query>.

The following are examples of SPL searches before and after SPL optimization:


Use case	Input	GPT Output	Llama Output
Select correct index	`sourcetype=access_combined status=404 \| stats count by clientip`	`index=web_logs sourcetype=access_combined status=404 \| stats count by clientip`	`index=web sourcetype=access_combined status=404 \| stats count by clientip`
	`search index=* error \| stats count by host`	`index IN (_audit, _introspection, _cmc_summary, _telemetry, summary) message="error" \| stats count by host`	`search index=main sourcetype=error \| stats count by host`
	`index=* \| fields _time userIdentity.arn eventName`	`index="aws" sourcetype="aws:cloudtrail" \| fields _time userIdentity.arn eventName`	`search index=_internal sourcetype=stash earliest=-1h@h latest=-0h@h \| fields _time userIdentity.arn eventName`
Optimize stats > tstats	`index=_internal sourcetype=* \| stats count by host, sourcetype`	`\| tstats count FROM index=_internal WHERE sourcetype IN ("splunkd", "splunk_web_access") BY host, sourcetype`	`\| tstats count WHERE index=_internal BY host, sourcetype`
Optimize stats > tstats	`index=_internal sourcetype=* \| stats latest(_time) as last_seen by sourcetype`	`\| tstats latest(_time) as last_seen FROM index=_internal WHERE isnotnull(sourcetype) BY sourcetype<pre>`	`\| tstats latest(_time) AS last_seen WHERE index=_internal BY sourcetype`
Wildcard replacement	`index=* sourcetype=audittrail user=*admin \| stats count by action`	`index=_audit sourcetype=audittrail user=*admin \| stats count by action`	`\| tstats count WHERE index=audittrail AND sourcetype=audittrail AND user=*admin BY action`

Use case	Input	GPT Output	Llama Output
Select correct index	`sourcetype=access_combined status=404 \| stats count by clientip`	`index=web_logs sourcetype=access_combined status=404 \| stats count by clientip`	`index=web sourcetype=access_combined status=404 \| stats count by clientip`
	`search index=* error \| stats count by host`	`index IN (_audit, _introspection, _cmc_summary, _telemetry, summary) message="error" \| stats count by host`	`search index=main sourcetype=error \| stats count by host`
	`index=* \| fields _time userIdentity.arn eventName`	`index="aws" sourcetype="aws:cloudtrail" \| fields _time userIdentity.arn eventName`	`search index=_internal sourcetype=stash earliest=-1h@h latest=-0h@h \| fields _time userIdentity.arn eventName`
Optimize stats > tstats	`index=_internal sourcetype=* \| stats count by host, sourcetype`	`\| tstats count FROM index=_internal WHERE sourcetype IN ("splunkd", "splunk_web_access") BY host, sourcetype`	`\| tstats count WHERE index=_internal BY host, sourcetype`
Optimize stats > tstats	`index=_internal sourcetype=* \| stats latest(_time) as last_seen by sourcetype`	`\| tstats latest(_time) as last_seen FROM index=_internal WHERE isnotnull(sourcetype) BY sourcetype<pre>`	`\| tstats latest(_time) AS last_seen WHERE index=_internal BY sourcetype`
Wildcard replacement	`index=* sourcetype=audittrail user=*admin \| stats count by action`	`index=_audit sourcetype=audittrail user=*admin \| stats count by action`	`\| tstats count WHERE index=audittrail AND sourcetype=audittrail AND user=*admin BY action`