Analyze service performance with Tag Spotlight

Use Tag Spotlight to analyze the performance of your services and to discover trends that contribute to high latency or error rates.

For every service, Tag Spotlight provides time-series charts for request, error, and duration (RED) metrics for every value of the span tags you've indexed. Request and error charts display the total number of requests, errors, and root cause errors, and duration charts display p50, p90, and p99 latency. These values are based on Troubleshooting MetricSets (TMS), which Splunk RUM generates for every indexed span tag.

When you select span tag values or a time range, you can view relevant traces to learn more about outlying incidents. For example, the following image shows a chart for requests and errors and a chart for duration for a sample frontend service. Under the charts there are bar charts with RED metrics for each indexed span tag.

This image shows requests and errors and latency charts as well as requests and errors broken down by indexed span tag for an example frontend service.

To learn how to index span tags to generate Troubleshooting MetricSets, see Index span tags to create Troubleshooting MetricSets. For a general overview of MetricSets, see Learn about MetricSets.

View service performance by indexed span tags

To view service performance broken down by your indexed span tags, follow these steps:

  1. Navigate to Tag Spotlight through any of these paths:
    • On the APPLICATION SUMMARY DASHBOARD, expand the service you're interested in and select see all.

    • In the service view, select the Tag Spotlight tab.

    • Use the search in the top toolbar, search for Tag Spotlight and select the navigation result.

  2. Add time range, environment, workflow, service, operation, and tag filters as need to refine the data in your Tag Spotlight view. The default time range is for the last 15 minutes, and the data resolution is 10 seconds.

  3. Use the request and errors and latency time-series charts at the top to see a distribution of your RED metrics.

  4. Use the bar charts under the request and errors and latency time-series charts to view RED metrics for each indexed span tag.

  5. View the distribution of all indexed span tags. The tag bar charts display either request and error distributions or latency distribution. Use the Cards display menu to select the data you want to display in the bars.

  6. Select the menu on the top left of the bar chart section to select which metrics to display in each tag panel. You can also use this menu to select whether to display tags with no values.

Use the histogram to visualize Troubleshooting MetricSets

The histogram visualizes the following frontend metrics (metrics that have P values):

  • Core Web Vitals:

    • Largest Contentful Paint (LCP)

    • Cumulative Layout Shift (CLS)

    • Interaction to Next Paint (INP)

  • Page Load/Route Change Duration

You can switch the histogram view for these metrics. Select a bucket to drill down to the sessions belonging to the bucket. Select the Sessions tab to drill down to the sessions belonging to the bucket.

Note: All sessions are exemplar sessions.

Customize the card display order and visibility

To configure the layout of the cards on the Tag Spotlight page, follow these steps:

  1. Select the filter icon in the top left corner of the cards section, and select Customize card display order.

  2. To reorder cards, drag them up or down in the list. Arrange the tags by priority, order of importance, or other criteria.

  3. To change the visibility of a card, select a value from its Display options pull-down menu.

  4. Select Save.

To understand the source of an incident use 1 of these options to drill down into indexed span tags for a selected service:

  • Use the filter bar to filter the requests shown in Tag Spotlight by environment, service, and operation. You can also create a custom filter based on indexed or unindexed tags. The filter context is preserved even when you navigate away from the page.

  • Select the menu on the top left of the bar chart section to select which metrics to display for any indexed span tag.

  • Select a span tag value then select Add to filter to filter the RED metrics charts to spans containing that value.

  • Hover over the RED metrics charts to view a summary of metrics for the specified time. The resolution of data for the RED metrics chart is 10 seconds.

  • Select within the RED metrics charts to load example traces for the time you selected.

  • Select a span tag card header to add the top 5 span tag values in the card to the RED metrics charts.

  • Control what appears in the RED metrics chart by showing or hiding metrics in the chart’s legend. For example, you can select the eye icon beside Requests to view only errors.

Learn more

To view a detailed examples using Tag Spotlight, see: