Configure your Splunk Observability Cloud account to collect GCP VertexAI metrics

Learn how to configure your Splunk Observability Cloud account to collect GCP VertexAI metrics.

You can monitor the performance of Google Cloud Platform (GCP) VertexAI applications by configuring your GCP VertexAI applications to send metrics to Splunk Observability Cloud. This solution creates a cloud connection in your Splunk Observability Cloud account that collects metrics from Google Cloud Monitoring.

Complete the following steps to collect metrics from and monitor GCP VertexAI.

  1. Connect GCP to Splunk Observability Cloud. For more information on the connection methods and instructions for each method, see Connect to Google Cloud Platform.
  2. Run your app.py file.
    You can call your GCP VertexAI models within your application either on local or on cloud. The following command calls a local Python application file with model invocations.
    python app.py
  3. Use the GCP VertexAI navigator to monitor your GCP VertexAI applications. For more information on navigators, see Use navigators.
    1. From the Splunk Observability Cloud main menu, select Infrastructure.
    2. Under AI/ML, select AI Frameworks.
    3. Select the GCP VertexAI summary card.

Metrics

Learn about the available metrics for GCP VertexAI.

The following metrics and resource attributes are available for GCP VertexAI applications. For more information on these metrics, see Cloud Monitoring metrics for Vertex AI in the Google Cloud documentation.
Metric nameUnitDescription
prediction/online /prediction_countcountNumber of online predictions.
prediction/online /prediction_latenciesmsOnline prediction latency of the deployed model.
prediction/online /response_countcountNumber of different online prediction response codes.
prediction/online /prediction_latencies.countcountNumber of online predictions.
prediction/online /prediction_latencies.sumOfSquareDeviationmsThe sum of squared deviation for prediction latencies.
publisher/online_serving /model_invocation_countcountNumber of model invocations (prediction requests).
publisher/online_serving /model_invocation_latencies.sumOfSquareDeviationmsThe sum of squared deviation for model invocation latencies.
publisher/online_serving /model_invocation_latencies.countcountNumber of model invocations (prediction requests).
publisher/online_serving /model_invocation_latenciesmsModel invocation latencies (prediction latencies).
publisher/online_serving /token_countcountAccumulated input/output token count.
publisher/online_serving /consumed_token_throughputcountOverall throughput used (accounting for burndown rate) in terms of tokens.
publisher/online_serving /consumed_throughputcountOverall throughput used (accounting for burndown rate) in terms of characters.
publisher/online_serving /character_count count

Accumulated input/output character count.

publisher/online_serving /first_token_latenciesmsDuration from request received to first token sent back to the client.
publisher/online_serving /first_token_latencies.countcountNumber of first token latencies.
publisher/online_serving /first_token_latencies.sumOfSquareDeviationmsThe sum of squared deviation for first token latencies.

Attributes

Learn about the available resource attributes for GCP VertexAI.

The following resource attributes are available for all GCP VertexAI metrics:

  • gcp_project_status

  • gcp_project_name

  • gcp_project_label_last_revalidated_by

  • model_user_id

  • gcp_project_number

  • request_type

  • gcp_id

  • gcp_project_label_cloud_registration_id

  • gcp_project_creation_time

  • gcp_project_label_last_revalidated_at

  • input_token_size

  • output_token_size

  • project_id

  • metricTypeDomain

  • gcp_project_label_environment

  • publisher

  • monitored_resource

  • gcp_project_label_account_type

  • gcp_project_label_owner_group

  • service

  • Location

In addition, the type resource attribute is available for the publisher/online_serving /token_count and publisher/online_serving /character_count metrics.

Troubleshoot

Learn how to get help if you can't see your data in Splunk Observability Cloud.

If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways:

  • Prospective customers and free trial users can ask a question and get answers through community support in the Splunk Community.