--- sourceDocument: Australia Enable AI sourceDocumentLink: https://www.servicenow.com/docs/r/australia/intelligent-experiences Release : - australia ft:locale : - en-US ft:publication_title : - Australia Enable AI ft:clusterId : - platai bundleId : - platai workflow : - Platform --- # Monitoring an AI system # Monitoring an AI system {#ariaid-title1} Release version: Australia Updated September 10, 2026 ![](https://www.servicenow.com/docs/portal-asset/ico-clock) 6 minutes to read Summarize ![AI sparkle icon](https://servicenow.com/docs/portal-asset/ai-sparkle-icon) Summarized using AI This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete. ## Summary of Monitoring an AI system This feature enables ServiceNow customers to assess the quality and safety performance of individual AI systems by reviewing detailed scores, metric breakdowns, trends, and evaluated sessions in a centralized view. It focuses on a single AI system, providing insights without the need to navigate portfolio-level views, thus helping customers quickly identify and address issues specific to that system. Show full answer Show less ## Key Features * **Monitor Tab:** Access detailed scores, trends, and session data for a chosen AI system directly from its asset page. Ideal when you know which system requires attention. * **Score Cards:** View composite quality and safety scores with color-coded indicators (red, orange, green) for quick status identification. Each card includes metric contributions and detailed calculations. * **Metrics Evaluated:** See all metrics currently scored for the AI system, including their category (quality or safety) and sample rate, ensuring transparency on what is being measured. * **Performance Metrics:** Monitor average latency per session and average tokens per session to evaluate system responsiveness and estimate operational costs. * **Trend Analysis:** Track metric trends over time with clear visual distinctions between contributing and non-contributing metrics to detect regressions or improvements. * **Session Insights:** Review detailed lists of evaluated sessions with scores and performance data. Select any session to investigate its traces and spans for root cause analysis. * **Customization:** Add or remove evaluation metrics for specific AI systems without altering the global configuration, allowing tailored monitoring aligned with business priorities. ## Using Monitoring Effectively * Activate evaluation scoring for your AI system type (ServiceNow AI or external AI) and enable evaluation from the inventory to start monitoring. * Access monitoring via AI Control Tower through inventory or insights dashboards to select and analyze a specific AI system. * Use the score cards and metric trends to quickly identify which metrics impact quality and safety scores and investigate any declines or improvements. * Leverage the session details to perform in-depth investigations on specific interactions, understanding latency, token usage, and scoring outcomes. * Customize the monitored metrics to align with your organization's evaluation needs, focusing on the most relevant quality and safety indicators. ## Benefits for ServiceNow Customers * Gain focused visibility into individual AI system performance, enabling faster and more precise remediation of issues. * Understand metric-level impacts on overall quality and safety, supporting data-driven decisions to optimize AI reliability and compliance. * Detect regressions early through trend monitoring, helping maintain consistent AI quality and safety standards. * Streamline investigation workflows by accessing all related session data and trace details in one place. * Control evaluation scope flexibly by configuring metrics per AI system, ensuring monitoring aligns with your operational priorities. Assess the quality and safety performance of a specific AI system by reviewing its scores, metric breakdowns, trends, and evaluated sessions. ## Key benefits {#mon-ai-asset-monitor__key-benefits} * See how a single AI system is performing without navigating between portfolio-level views and session lists. * Identify which metrics are pulling a quality or safety score up or down by reviewing the data sources table directly on each score card. * Detect regressions specific to an AI system by tracking metric trends over time. * Review all evaluated sessions for an AI system in one place and open any session to investigate its traces and spans. {#mon-ai-asset-monitor__ul_benefits} The Monitor tab on the AI system asset page focuses on a single AI system. While the monitoring overview shows portfolio-wide scores and rankings, the Monitor tab shows scores, trends, and sessions for the selected system only. Use the Monitor tab when you already know which system needs attention and want to understand its performance in detail. To start monitoring scores for an AI system, you must activate evaluation scoring for the relevant system type (ServiceNow AI or external AI) and then enable evaluation for the AI system from the inventory. * [Activate evaluation scoring for ServiceNow AI systems](https://www.servicenow.com/docs/hW6XCW4UHVvy4ZsacqxNcQ "Assess the quality of your ServiceNow AI systems by activating evaluation scoring in AI Control Tower.") * [Activate evaluation scoring for external AI systems](https://www.servicenow.com/docs/DJwf5i7uHymbQr5MjNM7DA "Assess the quality and safety of your external AI systems by connecting them to AI Control Tower and activating evaluation scoring.") * [Enable evaluation for an AI system](https://www.servicenow.com/docs/gVJS0Rlr9yz3Xx7xczsG4w "Start generating quality and safety scores for one or more AI systems by enabling evaluation from the inventory.") Figure 1. Monitor tab on the AI system asset page ## Accessing monitoring details for an AI system {#mon-ai-asset-monitor__accessing} Access monitoring details for an AI system in one of the following ways: * Navigate to AllAI Control TowerHomeInventory. Select the AI system asset and then select the Monitor tab. * Navigate to AllAI Control TowerHomeInsightsMonitor and select a system name in the AI systems ranked by score widget. * Navigate to AllAI Control TowerHomeInsightsMonitorEvaluated sessions and select the AI system name in the AI system column. {#mon-ai-asset-monitor__ul_accessing} ## Quality and safety score cards {#mon-ai-asset-monitor__score-cards} The Average overall quality score and Average overall safety score cards display composite scores for this AI system, averaged across all evaluated sessions within the selected date range. Each score card shows a percentage, the percentage change from the prior period, and a trend chart. The data sources table lists every metric that contributes to the composite, along with its weight and current value. Scores are color coded for quick interpretation: * Red (0 to 50%): Low. Investigate immediately. * Orange (51 to 75%): Fair. Review recent sessions for trends. * Green (76 to 100%): Good. Meeting quality or safety targets. {#mon-ai-asset-monitor__ul_color_coding} ## Score card side panel {#mon-ai-asset-monitor__score-card-side-panel} Select the side panel icon on a score card to open a detailed breakdown of how the composite score is calculated. * See how the score is calculated by expanding the How is this score calculated? section, which displays a stacked bar visualization of metric contributions. Each segment's width represents the metric's weight, and its shading represents the metric's score. * Review the Nuances to consider section for notes about how the average score is derived and how different scoring formats (binary, percentage, tiered) affect the calculation. {#mon-ai-asset-monitor__ul_side_panel} For details on how scores are calculated, see [How evaluation scoring works](https://www.servicenow.com/docs/lPJ171BGrbCTJBIWy_AzRA "Understand how AI Control Tower calculates quality and safety scores so you can interpret results accurately and configure scoring to reflect your priorities."). ## Metrics evaluated {#mon-ai-asset-monitor__metrics-evaluated} View every metric currently scored for this AI system, whether the metric comes from your organization's global metric configuration or was added specifically for this system in the Metrics evaluated list. View the list to confirm exactly what's being evaluated for a single AI system without checking the global configuration separately. Metrics : Name of the metric. Category : Whether the metric is a Quality or Safety metric. Sample rate : Percentage of eligible sessions that this metric evaluates. This value is set at the metric level and applies wherever the metric is used. {#mon-ai-asset-monitor__dl_metrics_evaluated_columns} AI stewards can add or remove metrics for any AI system. Asset owners can do the same, but only for the AI systems they manage. For details on adding or removing metrics, see [Configure metrics evaluated for an AI system](https://www.servicenow.com/docs/kvefhLAjdODauUoQNsCTKA "Add or remove metrics for a specific AI system to tailor evaluation coverage without changing your organization's global metric configuration."). ## Avg latency per session {#mon-ai-asset-monitor__avg-latency-per-session} Determine whether recent agent activity or another change is affecting this AI system's response times by reviewing the Avg latency per session card and its change from the prior period. This value reflects the AI system's language model response time, not the total time to complete a trace or session. Select the expand icon on the card to open a chart of total latency by day, summed across all sessions evaluated that day. ## Avg tokens per sessions {#mon-ai-asset-monitor__avg-tokens-per-session} Estimate the cost of running this AI system by reviewing the Avg tokens per sessions card and its change from the prior period. This value averages the combined input and output tokens consumed per session over the last month. ## Monitor agent activity {#mon-ai-asset-monitor__trend-chart} Track how quality and safety metrics for this AI system are trending over the selected date range. The chart plots every metric with data for this AI system so you can detect regressions specific to it, confirm that recent changes improved performance, and see which metrics are shaping its overall scores. Two line styles distinguish how each metric contributes to scoring: * Solid lines represent metrics that contribute to this AI system's overall quality or safety score. These metrics are configured in a metric template. * Dotted lines represent metrics that are being collected for this AI system but don't contribute to its overall quality or safety score. These metrics aren't configured in any metric template. {#mon-ai-asset-monitor__ul_line_types} To add a dotted-line metric to your scoring formula or to adjust how much existing metrics contribute to your overall scores, see [Configure an evaluation metric template](https://www.servicenow.com/docs/~6Wl1cgX0wLnQujKladjaw "Adjust the scoring formula for a metric template to change which metrics contribute to quality or safety scores, or to change how much weight each metric carries."). Choose which metrics to display by selecting an option from the list. Lowest performing metrics : Show the top five metrics with the lowest average score over the selected time period. This is the default selection. Top performing metrics : Show the top five metrics with the highest average score over the selected time period. Quality metrics : Show only metrics in the Quality category. Safety metrics : Show only metrics in the Safety category. All metrics : Show every quality and safety metric that has data for the selected date range. {#mon-ai-asset-monitor__dl_trend_options} Alternatively, select an individual metric from the list to view performance for that metric over time. Point to a data point on the chart to see the exact score for that date. ## Recent evaluated sessions {#mon-ai-asset-monitor__recent-evaluated-sessions} The evaluated sessions table lists all sessions for this AI system within the selected date range. Scores are color coded for quick scanning. Select a session name to open the session detail page and begin investigating traces and spans. For details on investigating sessions, see [Session details](https://www.servicenow.com/docs/83h9a2Q3nbS~U6q~BdDn0w "View metric breakdowns, the judge's reasoning, and trace and span data for a specific evaluated session."). Name : Session identifier. Select the name to open the session detail page. Trace : Number of traces in the session. Spans : Number of spans across all traces in the session. Quality score : Composite quality score for the session, calculated from the weighted metrics in your quality template. Safety score : Composite safety score for the session, calculated from the weighted metrics in your safety template. Start time : When the session began. End time : When the session ended or timed out. Total runtime tokens : Combined input and output tokens consumed by the AI system during this session. This reflects the cost of running the interaction, not the cost of evaluating it. Latency (ms) : Combined language model response time, in milliseconds, across all traces in the session. This measures time spent in language model calls, not the total time to complete the session. {#mon-ai-asset-monitor__dl_session_columns} * **[Review scores for an AI system](https://www.servicenow.com/docs/sTFf3R~aYD8u2Vtd2x0Otw)** Determine whether a specific AI system is meeting quality and safety targets by reviewing its scores, identifying which metrics are affecting performance, and checking for regressions over time. * **[Configure metrics evaluated for an AI system](https://www.servicenow.com/docs/kvefhLAjdODauUoQNsCTKA)** Add or remove metrics for a specific AI system to tailor evaluation coverage without changing your organization's global metric configuration. *[\>]: and then