Continuous controls monitoring in the AI Risk and Compliance Workspace
Summarize
Summary of Continuous controls monitoring in the AI Risk and Compliance Workspace
The AI Risk and Compliance Workspace enables continuous controls monitoring (CCM) for AI systems, providing risk and compliance managers with real-time insights into control compliance. This functionality automates control verification using scheduled indicators, reducing manual testing efforts and improving visibility into control health. When an indicator fails, a GRC issue is automatically created, prompting remediation and impacting the compliance score of the affected AI asset.
Show less
Key Features
- Indicator-based monitoring: Controls are continuously assessed using indicators linked to AI systems, extending traditional GRC workflows to AI governance.
- Integration with AI evaluation frameworks: Organizations can connect external AI evaluation providers, such as Traceloop, to leverage specific AI metrics (e.g., toxicity, PII detection, prompt injection, agent goal accuracy) for control assessment.
- Compliance scoring and audit trails: Continuous monitoring supports compliance scoring for AI-specific control objectives and regulatory requirements, maintaining a centralized system of record for governance evidence.
- Automated issue creation: Non-compliance detected by indicators triggers automatic issue creation for prompt remediation by product owners.
- Flexible monitoring frequency: AI system controls can be evaluated daily, weekly, or monthly throughout the AI system’s lifecycle.
Practical Application for ServiceNow Customers
ServiceNow customers managing AI governance can leverage this capability to automate and enhance the monitoring of AI system controls, ensuring continuous compliance and reducing risks associated with manual testing gaps. By integrating evaluation frameworks and configuring compliance evaluations, customers gain comprehensive visibility into AI model risks and maintain audit-ready evidence aligned with internal policies and external regulations. This proactive approach helps prevent costly compliance violations and supports operational resilience.
Indicators in AI Risk and Compliance Workspace continuously monitor control compliance for AI systems.
Risk and compliance managers can continuously monitor the controls attached to AI systems. Connect an external AI evaluation framework to the existing indicators framework in AI Risk and Compliance Workspace.
Indicators for AI systems
In organizations managing Governance, Risk, and Compliance (GRC) workflows for AI systems, teams can be challenged when monitoring control effectiveness in real time. Manual control testing can leave gaps in visibility and increase the risk of undetected control failures. Without continuous insight into control performance, organizations can face delayed issue resolution, costly compliance violations, and reduced operational resilience. Continuous Controls Monitoring (CCM) automates control verification, reducing manual testing burden and providing real-time visibility into control health. AI Risk and Compliance Workspace evaluates whether a control is compliant or non-compliant using indicators that run on a schedule. When an indicator fails, a GRC issue is created so that the product owner of the affected asset can remediate it. The compliance score of the associated record decreases. This same indicator concept is extended to AI systems by using evaluation scores produced by an AI evaluation framework.
Value for AI governance
Applying indicators to AI system entities gives an organization a single system of record for AI governance evidence. This approach enables continuous monitoring of model risk instead of relying on periodic manual reviews. It also provides an audit trail that supports Compliance Scoring for AI-specific control objectives and authorities, such as internal AI policies or external AI regulation.
AI evaluation framework providers
A compliance evaluation configuration can evaluate an AI system using metrics from ServiceNow or from other evaluation framework providers, such as Traceloop. Each provider exposes a different set of base metrics, such as toxicity, PII detection, prompt injection, and agent goal accuracy. The provider selected in a configuration determines which metrics are available.
For more information, see Create a compliance evaluation configuration, Use a compliance evaluation on an AI system record, and AI evaluation base metrics.Monitoring lifecycle
After an AI use case is deployed, it moves into a monitor state. Its controls are evaluated continuously over the lifetime of the AI system, on a daily, weekly, or monthly frequency. Each evaluation checks the trace, session, or span records produced by the evaluation framework against the configured metric thresholds. If a control fails, an issue is raised against the AI system and its compliance score decreases; when the control becomes compliant again, the score increases.