Create a compliance evaluation configuration

  • Release version: Australia
  • Updated July 25, 2026
  • 2 minutes to read

    • Create a compliance evaluation configuration that defines which AI systems, control objectives, metrics, and schedule a compliance evaluation applies to.

    Before you begin

    Role required: sn_grc_ai_gov.ai_risk_and_compliance_manager

    About this task

    A compliance evaluation configuration monitors AI systems for compliance and updates evaluation scores on an ongoing basis. Use evaluation frameworks from ServiceNow and Traceloop to produce trace, session, or span records for your AI systems. These frameworks help you derive a comprehensive evaluation of your AI systems.

    Procedure

    1. Navigate to All > AI Risk and Compliance > AI Risk and Compliance Workspace.
    2. Select the list icon .
    3. From the Controls monitoring module, select the Compliance evaluation configurations and select New.
    4. Define the evaluation details and data scope details.
      Table 1. Evaluation details and data scope fields
      Field Description
      Basic details
      Evaluation name A unique, descriptive name for this evaluation, using specific terms that identify the AI system, assessment scope, or version being evaluated (for example, Monitoring AI Toxicity).
      Description The purpose and scope of this evaluation, including what is being assessed, why it matters, and any key constraints or context users should know.
      Data scope details
      Authority documents The authority documents that govern this evaluation. The following options are provided:
      • NIST AI Risk Management Framework
      • EU Artificial Intelligence Act
      • AI Content Safety & Toxicity Standard
      Policies The policies that this evaluation aligns with or enforces. The following options are provided:
      • Artificial Intelligence Software Development Lifecycle Policy
      • Enterprise Artificial Intelligence Governance Policy
      • Internal use of AI Systems
      AI system type The category of AI system being evaluated. The following options are provided:
      • Agentic AI
      • Generative AI
      Provider The provider. The following options are provided:
      • ServiceNow
      • Others
      Metric category The metric category used in this evaluation:
      • Safety
      • Security
      • Quality
      Form showing compliance evaluation configuration with fields for evaluation details and data scope. The fields shown include options for selecting authority documents, policies, AI system type, and metric category.
    5. Select Next.
    6. Select Add control objectives to add control objectives to the scope.
    7. Map control objectives from the filtered list by selecting the check box of each desired control objective.
    8. Select Add to complete the control objective mapping, and then select Next.
      Form showing a table of compliance control objectives with checkboxes to select controls for evaluation scope.
    9. Define the evaluation criteria and frequency in the Evaluation criteria and frequency section.
      Table 2. Evaluation criteria and frequency fields
      Field Description
      Frequency The frequency at which evaluations are conducted. The following options are provided:
      • Daily
      • Weekly
      • Monthly

      For example, for AI toxicity, a monthly evaluation monitors control drift.
      Success condition result The result, Passed or Failed, assigned to an AI system when the evaluation criteria is met.
    10. In the All Evaluations section, define the conditions and the metrics under which the configuration is applicable.
      If multiple conditions are specified, they are evaluated in the order listed to determine applicability.
      Table 3. Conditions and metric thresholds fields
      Field Description
      Name A name for the evaluation.
      Conditions The field, operator, and value that filter which records this configuration applies to, for example, [Risk classification] [is] [High].

      Select "and" or "or" to combine multiple conditions, and select New condition set to add a condition set that is evaluated independently of other condition sets.
      Metric thresholds
      Metric The metric, operator, and threshold value that determine when a control is evaluated as compliant or non-compliant, for example, [Toxicity] [is] [True].

      Select Add metric to define additional metric thresholds for this configuration.
      Compliance evaluation configuration form showing the final step of the Configuring evaluation. The form displays evaluation conditions and metric thresholds.
    11. Select Submit.

    What to do next

    The compliance evaluation configuration is ready to be mapped to one or more AI systems. For information on mapping an evaluation configuration to AI systems, see Use a compliance evaluation on an AI system record.