Investigating low-scoring sessions

Narrow a quality or safety issue from a broad score to a specific cause by reviewing evaluated sessions and session details.

Investigation workflow

When a quality or safety score drops, investigate a session to narrow the problem from a broad score to a specific cause. Session-level scores tell you which interaction went wrong. Trace-level scores and the judge's reasoning tell you which exchange produced the problem. Span-level input and output data tells you exactly why: the wrong tool was called, a parameter was malformed, or a data source returned an error.

Important:

Trace and span detail for a session is available only for a limited time. To review the judge's reasoning and the input and output for a specific trace or span, investigate within 7 days for ServiceNow AI systems, or within 30 days for external AI systems. After that window, the session's quality and safety scores remain available, but the trace and span detail behind them is removed.

• Evaluated sessions
  View all evaluated AI sessions across your portfolio, compare quality and safety scores by AI system, and identify the sessions that need investigation.
• Session details
  View metric breakdowns, the judge's reasoning, and trace and span data for a specific evaluated session.
• Investigate a low-scoring session
  Determine the root cause of a low quality or safety score by working through the session detail, trace timeline, and span-level input and output data.