Now Assist Guardian analytics
Monitor the performance of guardrails enabled through Now Assist Guardian.
The Now Assist Guardian analytics dashboard helps admins monitor and evaluate the effectiveness of offensive content and prompt injection guardrails in tracking and analyzing requests sent to large language models (LLM) and their responses.
The indicators on the Now Assist Guardian dashboard page provide the following insights.
- Average latency as a result of active offensive content and prompt injection guardrails. High latency could mean increased guardrail activity in the period.
- Count and percentage of offensive content and prompt injection occurrences.
- Skills where offensive content and prompt injection occurrences were detected.
Apply the filters on the dashboard to view guardrail activity for skills in a date range. See Now Assist Analytics dashboard indicator details for information on the data and calculations behind each indicator.
Offensive content indicators
- Guardrail-added latency
- This area of the dashboard shows the average latency as a result of the active offensive content guardrail for the selected skills and date range.
Abbildung : 2. Guardrail-added latency indicator - Percentage flagged as offensive
- This area of the dashboard shows the percentage of requests and responses to and from the LLM service that are flagged for offensive content.
Abbildung : 3. Percentage flagged as offensive indicator - Total offensive content occurrences
- This area of the dashboard shows the total number of offensive content occurrences for the selected skills and date range.
Abbildung : 4. Total offensive content occurrences indicator - Categories of offensive content
- This area of the dashboard shows a breakdown of offensive content occurrences by the categories. If content is deemed to be offensive under more than one category, for example, toxic and defamatory, the occurrence is counted
individually toward both the categories. For more information on offensive content categories, see Now Assist Guardian.
Abbildung : 5. Categories of offensive content indicator - Offensive content occurrences by skill
- This area of the dashboard shows the number of offensive content occurrences over time by the skills in which the content is detected.
Abbildung : 6. Offensive content occurrences by skill indicator
Prompt injection indicators
- Guardrail-added latency
- This area of the dashboard shows the average latency as a result of the active prompt injection guardrail for the selected skills and date range.
Abbildung : 7. Guardrail-added latency indicator - Percentage flagged as prompt injection
- This area of the dashboard shows the percentage of requests and responses to and from the LLM service that are flagged for offensive content.
Abbildung : 8. Percentage flagged as prompt injection indicator - Total prompt injection occurrences
- This area of the dashboard shows the total number of offensive content occurrences for the selected skills and date range.
Abbildung : 9. Total prompt injection occurrences indicator - Prompt injection occurrences by skill
- This area of the dashboard shows the number of prompt injection occurrences over time by the skills where prompt injection attempts were detected.
Abbildung : 10. Prompt injection occurrences by skill indicator