Joost v Poppel
ServiceNow Employee

Why Assignment Group Prediction Is Hard

If you've worked in ITSM for any length of time, you know the pain of misrouted incidents. A user submits a ticket with "Workday timesheet not saving" and it bounces between the Service Desk, the Application Development team, and the Software group before someone finally picks it up. Every misroute costs time; SLAs slip, fulfillers waste effort triaging instead of resolving, and end users lose trust in the process.

Assignment group prediction has always been one of the trickier classification challenges. Unlike category or subcategory (which map relatively cleanly to keywords), assignment groups require a deeper understanding of organisational context, historical routing patterns, and the actual capabilities of each team. A traditional rule-based approach quickly becomes a maintenance nightmare, and even Predictive Intelligence on its own can struggle with groups that have limited training data or overlapping responsibilities.

 

This article walks you through a multi-method approach that combines five ServiceNow technologies to tackle this problem from different angles, and then lets the LLM decide which answer is best:

  • Predictive Intelligence: classification model trained on historic data to predict assignment groups
  • AI Search: semantic search across similar resolved incidents
  • Generative AI: match incident descriptions against available assignment group descriptions
  • Now Assist Skill Kit: a custom skill that orchestrates the methods and prompts the LLM
  • AI Agent Studio: an agentic workflow that triggers the skill and writes the result back to the incident

The key idea is simple: don't implement a single approach. Gather evidence from multiple sources and let the LLM weigh the results.

 

Disclaimer: All configuration, screenshots, and code in this article were built and tested on a ServiceNow Zurich Patch 9 Hotfix 1 instance. Given how fast the AI capabilities on the platform evolve, the UI and specific configuration screens may look different on newer releases. However, the core architectural pattern, multi-method classification orchestrated through the Skill Kit and AI Agent Studio, remains applicable across versions.

 

Architecture Overview

Before diving into the implementation, here's the high-level flow:

  1. A new incident is created and triggers an agentic workflow in AI Agent Studio.
  2. The workflow orchestrator dispatches multiple AI Agents in parallel, including our custom Assignment Group Selector agent.
  3. The Assignment Group Selector agent calls a Now Assist Skill Kit skill, which in turn executes three methods simultaneously:
    • Method A: Semantic search on similar resolved incidents (via AI Search) to find relevant assignment groups
    • Method B: A Predictive Intelligence classification model trained on historical incident data
    • Method C: Direct description matching between the incident description and active assignment group descriptions
  4. The LLM evaluates all three results and selects the most appropriate assignment group with a confidence score.
  5. If the confidence score is 80% or higher, the agent automatically updates the incident's assignment group and logs the reasoning in the work notes.

Implementation of solution

Let's build this solution step-by-step.

 

Step 1: Set Up Predictive Intelligence Classification

The first building block is a Predictive Intelligence classification solution. This gives us the traditional ML-based prediction that forms one of our three evidence sources.

Navigate to Predictive Intelligence > Classification Solutions and create a new classification definition with the following settings:

  • Label: Incident Assignment
  • Name: ml_incident_assignment
  • Table: Incident
  • Output Field: Assignment group
  • Input Fields: Short description, Description
  • Type of solution: Workflow Classification (this removes the need to train a separate word corpus)

For the training data filter, use resolved/closed incidents from the last 12 months with quality data:

  • Active is false
  • Created on Last 12 months
  • Assignment group is not empty
  • Short description is not empty
  • Description is not empty
  • State is one of New, In Progress, On Hold, Resolved

Set the Processing Language to English (or your preferred language), keep the Default English Stopwords, and set Training Frequency to Every 7 days (or your preferred frequency).

Note: These settings are an example; check the parameters you need to set to point to a relevant set of training data in your environment. 

Screenshot 2026-06-22 at 12.27.39.png

Classification Solution Definition for Incident Assignment - configured to predict assignment groups using Workflow Classification

 

After training, review the Solution Visualization tab. The precision/recall scatter plot shows how well the model performs for each assignment group class. You'll likely notice that some groups (like Software or Network) cluster high on both axes, while others with limited training data perform poorly. This is exactly why we don't rely on Predictive Intelligence alone; the multi-method approach compensates for these gaps.

Screenshot 2026-06-22 at 12.59.32.png

Solution Visualization showing estimated precision and recall across 16 assignment group classes

 

Step 2: Build the Now Assist Skill Kit Skill

In Now Assist Skill Kit, we build a net new skill, that gathers evidence from three different sources and feeds them all into a single LLM prompt for an informed decision on the right assignment group.

 

2.1 Now Assist Skill Kit - Tool Flow

The skill consists of four tools that execute before the final prompt. You can control these tools on the Add tools tab in the Now Assist Skill Kit:

  1. fetchDetailsIncident (Script) - Retrieves the incident's short description and description
  2. fetchAssignmentGroups (Script) - Fetches all active ITIL assignment groups with their descriptions
  3. IncidentSearch (Retriever) - Semantic search for similar incidents via AI Search
  4. AssignmentGroupPrediction (Predictive Intelligence) - Calls the ml_incident_assignment PI model

All four tools feed into a single Skill Prompt called "Assignment group selector" that makes the final determination.

Screenshot 2026-06-22 at 13.10.26.png

The Now Assist Skill Kit tool flow

 

2.2 Tool: fetchDetailsIncident

This script tool retrieves the incident details based on the incident number input.

(function runScript(context) {
    var incidentNumber = context.getValue('incident_number'); 
    var gr = new GlideRecord('incident');
    gr.addQuery('number', incidentNumber);
    gr.query();

    var incident = {};

    if (gr.next()) {
        incident = {
            number:            gr.getValue('number'),
            short_description: gr.getValue('short_description'),
            description:       gr.getValue('description')
        };
    } else {
        incident = { error: 'Incident not found: ' + incidentNumber };
    }    
    return incident;
})(context);

 

2.3 Tool: fetchAssignmentGroups

This script fetches all active assignment groups that are either ITIL-typed or have no type set, returning the sys_id, name, and description for each. The description field is critical, it's what enables LLM matching of incident text against group descriptions.

 

A note on context window limits: The list of assignment groups returned by this script is fed directly into the LLM prompt alongside the incident details, similar incident results, and PI predictions. Every token counts. If your instance has hundreds of assignment groups, sending all of them will consume a large portion of the LLM's context window, potentially crowding out the more valuable evidence from the other methods, or hitting the token limit entirely. Scope this query to a realistic candidate set. In this example, I filtered to ITIL-typed groups. Another approach that works well is limiting to groups that have actually been assigned to incidents in the last 6 months. This keeps the list relevant and manageable.

(function runScript(context) {
    var groups = [];

    // Resolve the sys_id of the 'itil' group type (avoid hardcoding)
    var itilTypeSysId = '';
    var typeGR = new GlideRecord('sys_user_group_type');
    typeGR.addQuery('name', 'itil');
    typeGR.setLimit(1);
    typeGR.query();
    if (typeGR.next()) {
        itilTypeSysId = typeGR.getUniqueValue();
    }

    // Query: active=true AND (type IS EMPTY OR type CONTAINS itil)
    var gr = new GlideRecord('sys_user_group');
    gr.addActiveQuery();
    var qc = gr.addNullQuery('type');
    if (itilTypeSysId) {
        qc.addOrCondition('type', 'CONTAINS', itilTypeSysId);
    }
    gr.query();

    while (gr.next()) {
        groups.push({
            sys_id: gr.getUniqueValue(),
            name: gr.getValue('name') || '',
            description: gr.getValue('description') || ''
        });
    }

    return JSON.stringify(groups);
})(context);

 

2.4 Tool: IncidentSearch (AI Search Retriever)

This is a retriever tool that performs a semantic search against the incident table to find similar resolved incidents and their assignment groups. For this the incident table needs to be indexed by AI Search, more specifically a semantic index need to be set on the incident table. In case this is not done, only keyword search will be available as option. 

 

My configuration is as follows:

  • Search query: {{fetchDetailsIncident.output}} (uses the incident details from the first tool as the search input)
  • Search space type: Table-based
  • Table: Incident
  • Fields returned: incident.number, incident.short_description, incident.assignment_group
  • Limit: 10 (maximum documents returned)
  • Search Criteria: Hybrid (combines semantic and keyword matching)

Screenshot 2026-06-22 at 13.23.19.png

AI Search retriever configuration - table-based hybrid search on incidents

 

2.5 Tool: AssignmentGroupPrediction (Predictive Intelligence)

This tool calls the PI classification solution we set up in Step 1.

  • Name: AssignmentGroupPrediction
  • Type of solution: Workflow Classification
  • Solution label: Incident Assignment
  • Solution name: ml_incident_assignment

Screenshot 2026-06-22 at 13.26.54.png

Predictive Intelligence tool configuration

 

 

  • Tool inputs:
    • description (string) → {{fetchDetailsIncident.output.description}}
    • short_description (string) → {{fetchDetailsIncident.output.short_description}}
  • Top N results: 3

 

Screenshot 2026-06-22 at 13.27.05.png

Tool inputs passing incident description and short description

 

2.6 The Skill Prompt: Assignment Group Selector

This is the prompt that brings everything together. It receives the output from all four tools and instructs the LLM to evaluate each method, weigh the evidence, and produce a final recommendation with a confidence score.

## Role
You are an ITSM Assignment Group categorisation expert. Your task is to analyse an incoming
incident and select the most appropriate Assignment Group. You will use different techniques
to see which assignment group is most applicable, including a search for similar incidents,
using a pre-trained predictive intelligence model and matching descriptions of incidents and
fetched assignment groups. Based on the outcome of each method, you will recommend the most
appropriate assignment group.

## Context
The incoming incident that needs to be analysed consists of potentially a description and
short description:
- Incident Description: {{fetchDetailsIncident.output.description}}
- Incident Short description: {{fetchDetailsIncident.output.short_description}}

The search for similar incidents and their respective assignment groups is as follows:
 {{IncidentSearch.rag_results}}

The outcome of the predictive intelligence pre-trained machine learning model is as follows:
 {{AssignmentGroupPrediction.outputs}}

The available Assignment Groups to choose from are listed below:
{{fetchAssignmentGroups.output}}

## Analysis:
This is the incoming incident. Do the analysis in the following order:
1. Method A - See if there are similar resolved incidents found and if there is an
   assignment group match.
2. Method B - Check the outcomes from the Predictive Intelligence model, to see which
   assignment groups are predicted.
3. Method C - Check the fetched assignment groups with their descriptions, and select the
   most suitable assignment group based on the incident short description and description.
   Make note of how confident you are in the selected option.
4. Determine based on the above methods A, B and C, what would be the most suitable
   assignment group. Take into account:
   - Semantic similarity of incidents from search results
   - Confidence score of the predictive intelligence model results
   - Confidence of semantic match on incident and assignment group descriptions
   - If there are any matching sysIds between the three methods
5. Present a combined vision, suggestion for the assignment group that needs to be chosen
   for the incoming incident based on your analysis.

## Output
Provide the following output:
- Incident number:  {{incident_number}}
- Summary of the incoming incident in max 25 words
- Assignment Group: selected assignment group name
- Assignment Group ID: Assignment group sysID
- Confidence: for the selection on a 1 to 100 scale formatted in a percentage - eg. 98%.
- Reasoning: A note on the combined analysis that you've done.
DO NOT INCLUDE anything else in the output.

## MANDATORY Output Format
You should follow a strict JSON format when generating a response, which should look like
below example:
{ "Incident Number"   : "INC0012345",
  "Incident Summary"  : "My VPN is not working properly; numerous login attempts failing",
  "Assignment Group"  : "VPN Connectivity",
  "Assignment SysID"  : "8a5055c9c61122780043563ef53438e3",
  "Confidence Score"  : "96%",
  "Reasoning"         : "The confidence score of the predictive intelligence model was high
                         enough compared to other results" }

The power of this prompt lies in the structured multi-method analysis. The LLM doesn't just pick the first answer it gets. Instead it compares results across all three methods, looks for agreement (matching sysIds), weighs confidence scores, and produces an output including it's reasoning. When all three methods agree, you get a high confidence score. When they disagree, the LLM's reasoning explains why it favoured one over another.

 

Step 3: Set Up the AI Agent Studio Workflow

Now we wire everything together in AI Agent Studio using an agentic workflow.

 

3.1 Duplicate the OOTB Workflow

ServiceNow ships an out-of-the-box workflow called "Triage and categorize ITSM incidents" that already handles category, subcategory, service, and CI classification. Rather than building from scratch, we duplicate this workflow and extend it with a new assignment group agent.

 

Navigate to AI Agent Studio, find the "Triage and categorize ITSM incidents" workflow, and use the three-dot menu > Duplicate to create a copy.

Screenshot 2026-06-22 at 13.36.27.png

 

Rename the copy to "Triage and categorize ITSM incidents (Copy)" and update the Workflow description:

This agentic workflow enables fulfillers to identify the category, subcategory, service, offering, configuration item and assignment group for an incident automatically, and finally link a major incident or problem.

 

3.2 Update the Workflow Orchestrator Prompt

The orchestrator is the brain of the workflow. It coordinates all AI Agents in parallel and ensures each one runs to completion independently. The key updates from the OOTB version:

  • Added a fourth agent: Assignment Group Selector
  • Added isolation rules to prevent one agent's failure from affecting others

Note: The updated prompt I used was as follows, but you can run your own logic here. 

# ROLE
You are the ITSM Incident Triage Orchestrator. Your sole responsibility is to coordinate
the parallel execution of four independent enrichment AI Agents against a single incident
record and guarantee that every agent runs to completion, regardless of the outcome of any
other agent.

# OPERATING PRINCIPLES
- PARALLEL DISPATCH MANDATE: All four AI Agents are dispatched in a single parallel batch.
  Sequential execution is forbidden.
- INDEPENDENCE ISOLATION: Each AI Agent executes in isolation. The outcome of one agent
  (success, empty result, error, or timeout) NEVER influences the dispatch, execution, or
  completion of any other agent.
- NO CASCADE TERMINATION: Workflow termination is only permitted after all four agents have
  reached a terminal state. Early termination of the orchestrator is forbidden under any
  circumstance.
- SILENT INPUT VALIDATION: If the incident number is absent or malformed, terminate
  silently with status INPUT_INVALID. Do not prompt the fulfiller for additional details.
  The incident number is the only accepted input.
- DETERMINISTIC COMPLETION: Every agent must reach exactly one named outcome state before
  the orchestrator finalizes.
- NO INFERENCE OF DEFAULTS: Never substitute, retry with altered parameters, or infer
  values for an agent that returns empty. Empty is a valid terminal state.

# INPUT CONTRACT
- Required: `incident_number` (e.g., INC0010234)
- Forbidden: any additional fulfiller-supplied context. Ignore extra inputs if provided.

# NAMED OUTCOME STATES (per agent)
- AGENT_COMPLETED_WITH_RESULT — agent executed and returned a non-empty result
- AGENT_COMPLETED_EMPTY — agent executed and returned no actionable result (valid terminal)
- AGENT_FAILED — agent returned an error or exception
- AGENT_TIMEOUT — agent did not return within its execution window

All four states are TERMINAL and NON-BLOCKING for the orchestrator.

# FORBIDDEN PATTERNS
- Executing agents sequentially or in pairs
- Skipping any of the four agents under any condition
- Terminating the orchestrator after a single agent failure
- Re-prompting the fulfiller after an agent returns empty or fails
- Aborting remaining agents because an earlier agent succeeded "sufficiently"
- Inferring or fabricating results for any agent that failed or timed out

---

### Step 1: Retrieve and Validate Incident Number
Extract the `incident_number` from the fulfiller input. Verify it is present and conforms
to the expected incident identifier pattern.

### Step 2: Input Validation Gate
Evaluate input readiness:
- Confirm `incident_number` is present and non-empty
- Confirm the format matches an ITSM incident identifier
- Confirm no required input is missing
Generate an input validation report. If validation fails, terminate silently with status
INPUT_INVALID and emit no fulfiller-facing prompt. If validation passes, proceed to Step 3.

### Step 3: Dispatch All Four AI Agents in Parallel
Initiate the following four AI Agents as a single concurrent batch. All four calls must be
issued before awaiting any response.

1. **Categorize ITSM Incident AI Agent**
   - Purpose: Identify the appropriate category and sub-category for the incident.
   - Input: `incident_number`

2. **Classify Service and CI AI Agent**
   - Purpose: Determine the correct service, service offering, and configuration item (CI).
   - Input: `incident_number`

3. **Assignment Group Selector**
   - Purpose: Determine the most appropriate assignment group to handle the incident.
   - Input: `incident_number`

4. **Link Major Incident or Problem AI Agent**
   - Purpose: Identify any related Major Incident or Problem and link it to the current
     incident when a match is found.
   - Input: `incident_number`

Each dispatch is independent. Do not chain, await, or condition any dispatch on any other.

### Step 4: Dispatch Validation Gate
Verify dispatch completeness:
- All four agents were invoked in the same parallel batch
- No agent was omitted, deferred, or made conditional on another
- Each invocation received the validated `incident_number`
Generate a dispatch validation report. If any agent was not dispatched, dispatch the
missing agent(s) immediately before proceeding.

### Step 5: Collect Each Agent's Result in Isolation
Await the response from each AI Agent independently. Process each response in its own
isolated observation:
- Capture the agent's returned payload (if any)
- Assign exactly one terminal outcome state
- Record the outcome against the agent name

Critical isolation rule: an error, empty result, or timeout from any one agent MUST NOT
cause the orchestrator to abort, retry, or alter the collection of any other agent.

### Step 6: Independence Enforcement Gate
Examine the collection phase for isolation integrity:
- Confirm that each of the four agents has been assigned exactly one terminal outcome state
- Confirm no agent's outcome was skipped due to another agent's failure
- Confirm no early termination occurred

### Step 7: Aggregate Results
Compile the four agent outcomes into a single structured triage result. Do not synthesize,
merge, or cross-reference payloads between agents. Aggregation is concatenation, not
reconciliation.

### Step 8: Completion Validation Gate
Assess orchestration completeness:
- All four agents have a recorded terminal outcome state
- The aggregated result contains exactly four agent entries
- No agent entry is missing, duplicated, or fabricated

### Step 9: Present Final Triage Summary
Present the aggregated triage result to the workflow. Terminate the orchestrator.

 

The following four agents should be part of the workflow - where I created/added the Assignment Group Selector AI Agent to the other three AI Agents that come OOTB:

  1. Categorize ITSM Incident AI Agent - identifies category and sub-category
  2. Classify Service and CI AI Agent - determines service, offering, and configuration item
  3. Assignment Group Selector - our custom agent that determines the assignment group
  4. Link Major Incident or Problem AI Agent - links related major incidents or problems

NB. See section 3.5 how to create the Assignment Group Selector AI Agent.

Screenshot 2026-06-22 at 13.45.55.png

AI Agents in Agentic Workflow

 

3.3 Configure Security Controls

Set up the appropriate access controls for the workflow:

User Access (ACL):

  • Decision type: Allow if
  • Roles: itil

Data Access:

  • User identity type: AI user
  • AI user: itsm.aia.worker

3.4 Configure the Trigger

Set up the workflow trigger to fire automatically when new incidents are created:

  • Select a trigger: Created
  • Trigger name: Incident created
  • Trigger objective: Help me resolve ${number}
  • Table: Incident
  • Conditions: Active is true
  • User identity: Use an existing table → Caller [incident]
  • Channel: Now Assist Panel
  • Show an alert to users: Yes

Screenshot 2026-06-22 at 13.50.08.pngScreenshot 2026-06-22 at 13.50.36.png

Screenshot 2026-06-22 at 13.50.49.png

Trigger Agentic Workflow

 

3.5 Create the Assignment Group Selector Agent

Within the workflow, click Add AI agents > Create new AI agent to create the custom agent.

 

Agent Name and Description:

  • AI agent name: Assignment Group Selector
  • AI agent description: This agent streamlines incident management by intelligently assigning incidents to the most appropriate support group. It fetches incident data, predicts the best assignment group using an AI skill, and ensures accuracy by considering confidence scores and taking no action for low-confidence predictions.

Screenshot 2026-06-22 at 13.56.55.png

Name & Description AI Agent

 

Agent role:

The Incident Assignment Agent specializes in automatically determining and setting the correct assignment group for newly created incidents. It retrieves incident details, utilizes a predictive skill to suggest an assignment group with a confidence score, and will not execute assignment group updates if the confidence score is below a predefined threshold.

 

List of steps:

### Step 1: Use the provided incident number in calling tools in this AI Agent.

### Step 2: Predict the Assignment Group for the incident number.

### Step 3. Evaluate the confidence score of the assignment group determination.
Check the output of the assignment group determination, to be used in processing of further steps.

### Step 4. Update the assignment group
Based on the confidence score of the assignment group determination, update the assignment group in the incident:
- If the confidence score is less than 80%, DO NOT update the assignment group.
- If the confidence score is 80% or higher, assign the incident to the predicted assignment group.

### Step 5. Update incident work notes with a summary
- Generate a summary of the steps executed in this AI Agent up to this point.
- Update the incident work notes of the incident with the generated summary.

Key rules:
- NEVER ask the user for input, confirmation, or guidance at any point.
- NEVER use collect_input_from_user or show_output_to_user modes.

 

Note: The 80% confidence threshold is a design choice, tune it based on your organisation's tolerance for incorrect assignments. Higher thresholds mean fewer automatic assignments but higher accuracy; lower thresholds mean more automation but more potential misroutes. 

 

3.6 Agent Tools

The Assignment Group Selector agent has three tools, all set to Autonomous execution mode with Display output disabled.

 

Tool 1: Determine assignment group (Now Assist Skill)

This tool calls the Skill Kit skill we built in Step 2.

  • Select skill: Assignment Group Selector
  • Input: incident_number (string)
  • Tool name: Determine assignment group
  • Tool description: This tool will predict the right assignment group for the incoming incident
  • Execution mode: Autonomous

Screenshot 2026-06-22 at 14.07.26.png

Tool calling the Now Assist Skill Kit skill

 

Tool 2: Update incident assignment group (Record Operation)

This tool writes the predicted assignment group back to the incident record.

  • Tool name: Update incident assignment group
  • Tool description: Update the incident assignment group, based on the earlier determined assignment group
  • Inputs: incidentnumber (Number of the incident), assignmentgroup (The sysid of the assignment group)
  • Table: Incident
  • Operation: Update records
  • Conditions: Number is {{incidentnumber}} AND Active is true
  • Field values: Assignment group = {{assignmentgroup}}
  • Execution mode: Autonomous

Screenshot 2026-06-22 at 14.07.47.png

Screenshot 2026-06-22 at 14.08.04.png

Tool with record operation towards incident assignment group

 

Tool 3: Update worknotes (Script)

This tool logs the agent's reasoning and actions into the incident work notes for auditability.

  • Tool name: Update worknotes
  • Tool description: This tool updates the worknotes of the incident
  • Script inputs: incident_number (mandatory), worknotes (mandatory)
(function(inputs) {
    var incidentNumber = inputs.incident_number;
    var worknotes = inputs.worknotes;

    var gr = new GlideRecord('incident');
    gr.addQuery('number', incidentNumber);
    gr.query();

    if (gr.next()) {
        gr.work_notes = worknotes;
        gr.update();

        return {
            status: 'success',
            incident_number: incidentNumber,
            incident_sys_id: gr.getUniqueValue(),
            message: 'Work notes updated successfully on incident ' + incidentNumber
        };
    } else {
        return {
            status: 'error',
            incident_number: incidentNumber,
            message: 'No incident found with number ' + incidentNumber
        };
    }
})(inputs);

Screenshot 2026-06-22 at 14.08.31.png

Tool calling script for work notes updates

 

Step 4: Testing the End-to-End Flow

When running AI Agents and also using the Service Operations Workspace in ITSM, service desk agents can see what has happened from an AI Workflow perspective, as well as what fields for example have been modified by Now Assist. 

 

As soon as the incident is created, the agentic workflow triggers. You'll see the green sparkling AI icon in the top bar, and the AI Workflows panel on the right shows the workflow is active with "Checking on remaining steps."

img23_incident_during_workflow.jpeg

Service Operations Workspace showing AI Agents are working on the incident

 

Within approximately two minutes, the workflow completes. The result:

  • Assignment group is set to "Software" with the AI badge indicating it was auto-populated
  • Subcategory, Service, and Configuration item are also populated by the other OOTB agents
  • The work notes contain a detailed summary from the Assignment Group Selector agent, including the confidence score (99%) and the sys_id of the selected group

img25_ai_badges_closeup.jpeg

Fields that were predicted by the AI agents are clearly marked with a green ✦ AI badge, giving fulfillers immediate visibility into which values were auto-populated and prompting them to review for accuracy. Furthermore, on the right hand-side it is possible to open up the details of the AI workflows that have been executed, giving details on what has been going on in the background. I always update the work notes as well with activity, as this again is being used in other capabilities like summarization. 

Screenshot 2026-06-22 at 14.22.21.png

 

Inspecting the Decision Logs

For deeper insight into what happened, navigate to AI Agent Studio > Activity and find the executed workflow. The AI agent decision logs show the full reasoning chain for each agent.

Screenshot 2026-06-22 at 14.26.12.png

 

For the Assignment Group Selector, you can see the agent's thought process:

img29_now_assist_panel_history_1.jpeg

 

Key Takeaways

  • Don't rely on a single classification method. Predictive Intelligence works well for assignment groups with strong training data, but struggles with underrepresented groups. Semantic search catches patterns that the ML model misses. Description matching provides a fallback when historical data is sparse. Together, they produce a more robust prediction.
  • Now Assist Skil Kit. I used NASK because I can combine the probabilistic capabilities from the LLM with the more deterministic needs of calling specific tools in a certain order, before running the prompt. NASK allows you to build a very boxed down skill tailored to a specific need, like assignment group determination. 
  • Let the LLM score itself. By feeding all three results into a single prompt, the LLM can compare evidence, identify agreement, and make a reasoned choice, including a confidence score that you can use to decide whether to auto-assign or defer to a human.
  • Build on what's already there. Duplicating the OOTB triage workflow and adding your custom agent means you get category, subcategory, service, CI, and assignment group classification in a single automated run, without rebuilding an agentic workflow from scratch.
  • Make it auditable. The work notes activity stream and the AI agent decision logs give fulfillers and managers full visibility into why a particular assignment group was chosen. This builds trust in the automation and makes it easy to identify when the model needs tuning.
  • Set a confidence threshold. The 80% threshold provides a safety net. High-confidence predictions are applied automatically, while uncertain ones are left for human review. Adjust this based on your organisation's needs and the maturity of your training data.

What's Next

This method applied is not limited to assignment groups. You can apply the same multi-method approach to any classification field, eg. priority, service, CI, or even custom fields. The architecture of gathering evidence from multiple sources and letting the LLM weigh the results is a transferable pattern that scales across use cases.

If you want to take it further, consider adding additional methods to the skill, for example, querying a knowledge base for resolution patterns, or using business rules to apply domain-specific logic before the LLM makes its final call. For example, your knowledge base could serve as a dynamic set of rules, which is to be applied on your skill for assignment group determination, ie. if something changes, you update the KB article with out the need for updating the NASK prompt. 

 

Have questions or want to share how you've implemented this in your environment? Drop a comment below, I'd love to hear about your experience.

Comments
nsitbon
Kilo Explorer

thanks for the article:)

Version history
Last update:
4 hours ago
Updated by:
Contributors