---
sourceDocument: Australia Enable AI
sourceDocumentLink: https://www.servicenow.com/docs/r/australia/intelligent-experiences

 Release :

    - australia

ft:locale :

    - en-US

ft:publication_title :

    - Australia Enable AI

ft:clusterId :

    - platai

bundleId :

    - platai

workflow :

    - Platform


---

# Evaluation flow

# Evaluation flow {#ariaid-title1}

Release version: Australia  
Updated March 12, 2026  
![](https://www.servicenow.com/docs/portal-asset/ico-clock) 3 minutes to read
Summarize  
![AI sparkle icon](https://servicenow.com/docs/portal-asset/ai-sparkle-icon) Summarized using AI  
This content was generated using new OpenAI-powered functionality. Results are provided on an as is basis and are not guaranteed to be accurate or complete.  

## Summary of Evaluation Flow

The evaluation flow in ServiceNow automates the assessment of virtual agent conversations upon their completion.
It captures user interactions, triggers evaluations based on predefined conditions, and processes the data to generate evaluation metrics, ensuring efficient and effective conversation assessments.
Show full answer Show less  

## Key Features

* **Conversation Capture:** Logs all end-user interactions in the Conversation table and updates the record to 'Complete' when a conversation ends.
* **Automated Evaluation Trigger:** Fires when conditions are met, ensuring evaluations are performed only for a limited number of conversations daily.
* **Sampling Logic:** Evaluates only 10% of conversations through a random selection process.
* **Transcript Construction:** Builds detailed chat transcripts, including user messages, virtual agent responses, and knowledge articles referenced.
* **Chat Classifier:** Utilizes a classifier to determine if a conversation should be evaluated based on topic and category.
* **Skill Invocation:** Asynchronously calls the Now Assist Skill API for performance scoring, generating evaluation metrics with scores and reasoning.

## Key Outcomes

Customers can expect streamlined evaluation processes that enhance the quality of virtual agent interactions. The automated flow ensures that only relevant conversations are assessed, leading to insightful metrics that inform future improvements. By filtering out irrelevant interactions and focusing on key performance indicators, organizations can optimize service delivery and user satisfaction effectively.  
The workflow for evaluation execution, which performs evaluations when conversations are completed.
Conversations are evaluated using the following logic:

1. Conversation capture:

   All end-user interactions with the virtual agent are logged in the Conversation table \[sys_cs_conversation\]. When a user ends the conversation, the record's state is updated to Complete.
2. Automated flow evaluation trigger:

   Flow name: Execute Evaluation.  
   Trigger condition:
   * Table: Conversation table \[sys_cs_conversation\]
   * State: Complete
   * Device type: Web Client, Slack, Teams, Bot to Bot, Messenger
   {#ai-eval-flow__ul_rfv_whh_ngc}
{#ai-eval-flow__ol_rwz_phh_ngc}

Sequence of execution:  
Action 0: Check evaluations count for today

* Perform a query on evaluation table and to get record count.
* If record count is less than Max Number of evaluations per day, continue to Action 1, else end flow.
{#ai-eval-flow__ul_zn2_zpy_wgc}  
Action 1: evalExecuteCondition

* Invokes the evalExecuteCondition.executeEvaluation script Include with conversation reference.
* Generates a random number (1--100). Proceeds only if ≤10 (10% random sampling).
* Outcome: Returns true or false for further processing.
{#ai-eval-flow__ul_dgq_j3h_ngc}  
Action 2: Conditional Branch

* If true: Proceed to the next action.
* If false: Evaluation stops.
{#ai-eval-flow__ul_bb5_q3h_ngc}

Action 3: Lookup Interaction Table:

Matches the conversation's channel metadata with the interaction table to fetch related records.

Action 4: Application Scope Filter:

If the interaction's application scope doesn't include hr, continue.

Action 5: buildTranscript:  
Detailed Transcript Construction:

* Tags: \[User\]: For user messages, \[Virtual Agent\]: For virtual agent messages.
* For any referenced Knowledge article:
  * Pulls the complete article body to replace genius result, tagged with \[Virtual Agent\]: Help articles for user query: and delimited by Article_Start/Article_End.
  * If the Knowledge article is in HR scope/inaccessible, skip evaluation.
  * If the Knowledge article content is \>10,000 words: Truncate at 10,000.
  * Attached files (PDF/Word/Txt): Use genius result instead.
  {#ai-eval-flow__ul_o3c_mjh_ngc}
* For referenced Catalog Items:Extracts name, short description, description, annotated as \[Virtual Agent\]: Please choose one of the below options: with citation number.

* If the first message is to the live agent, or the live agent is invoked within the first 120 words: skip evaluation.
{#ai-eval-flow__ul_thx_jjh_ngc}  
Outputs:

* ExecuteEvaluation (true/false)
* Chat Transcript
* Knowledge articles or catalog items referred
* Sys_id of first live agent invocation (if any)
* List of skills to invoke (all evaluation skills for Evaluation dashboard)
* Additional evaluation logs
{#ai-eval-flow__ul_yvk_bkh_ngc}

Action 6: Conditional Branch:

If ExecuteEvaluation is true: Continue to Action 7.  
Action 7: Chat Classifier Eval

* Builds the initial transcript from sys_cs_message.
* Uses Chat topic classifier to determine:
  * Should the conversation be evaluated? (ExecuteEvaluation: true/false)
  * Topic Name
  * Category (IT/HR)
  {#ai-eval-flow__ul_zz2_2jh_ngc}
* If ExecuteEvaluation is true: Proceed to Action 6.
{#ai-eval-flow__ul_m5x_1jh_ngc}

Action 8: Create or Update Evaluation Record:  
Create a record on Evaluation \[sn_na_conv_eval_evaluation\] table with:

* Document Conversation: Conversation reference
* State: Processing
* Topic, Category, Knowledge article or catalog references, first live agent sys_id, type, user who initiated, message log
{#ai-eval-flow__ul_cjw_hkh_ngc}

Action 9: For each skill:

Repeats for each skill flagged in Action 6.  
Action 10: invokeApiDefinition

* Inputs: Skill name, conversation, transcript, evaluation id
* Calls Now Assist Skill API asynchronously.
* Post processing available in sys_generative_ai_response_validator, performs the following parsing:
  * Score
  * Reason for Score
  * Examples for the reasoning
  {#ai-eval-flow__ul_m5m_tmh_ngc}
* Parsed data is created on the Evaluation Metrics \[sn_na_conv_eval_evaluation_metrics\] table (Score, Reasons, Examples, and the entire reasoning for scoring \[Scratchpad\]).
{#ai-eval-flow__ul_xbx_nmh_ngc}

Action 11: Waits 7 seconds before continuing to the next skill.  
Special behavior and edge case handling:

* Sampling: Only 10% of conversations (randomly chosen) are evaluated.
* Channel Filter: Only Web, Slack, Teams, Bot to Bot, Messenger.
* Application Scope: Excludes records with _hr_ in the scope.
* Knowledge article controls: No evaluation for HR or inaccessible. Knowledge articles, limits on Knowledge article size, and file handling.
* First live agent invocation: Excludes conversations routed to the live agent at the start or within 120 words.
* The Request Completion skill is added as part of a business rule where the score is tagged as the lowest between Slot filling and Intent.
* The reason on the record is added as follows:

      if (Slot filling score > Intent score) {
      Intent reason is used
      } else if (Slot filling score < Intent score) {
      Slot filling reason is used
      } else {
      Both are used
      }

{#ai-eval-flow__ul_tlr_bnh_ngc}