From version 24.2.0, Python Agent supports OpenAI Moderation API. The moderation API will prompt the users if the queried statement is policy compliant or not. The Python Agent captures the following metrics:

Calls to moderations API

The number of times moderation API is used every minute. (OpenAI|moderations|Calls per minute)

Flagged category metrics

The number of times specific types of content, like hate speech, are flagged. (OpenAI|moderations|Flagged Calls by category|hate)

Flagged queries count

The total amount of flagged content. (OpenAI|moderations|Flagged queries)

Total queries count

The overall number of queries processed. (OpenAI|moderations|Total queries)