ServiceNow Otto for CSM - Automated Quality Assurance

FernandoCastro · ‎07-22-2026

Automatically score and coach every closed case with GenAI-generated quality reviews

Automated quality assurance reviews closed cases against a configurable rubric and generates a quality score with qualitative feedback for each one, replacing manual spot-checks with consistent, near-100% coverage. Managers and agents see the results in role-specific dashboards, and admins control which parameters count and how much each one is weighted.

Automated quality assurance is delivered by the Now Assist for Service Quality v1.0 app, which is auto-installed as part of Now Assist for CSM. You will not need to separately activate it, but it may appear under its own app name in Studio or the application list.

Key Capabilities

Automatic review on case close: The skill triggers when a case moves to Closed or Resolved, so every case gets scored without agents or managers doing anything.
Configurable rubric library: Nine base-system categories and 21 parameters cover issue understanding, communication quality, empathy, knowledge base use, first-contact resolution, and closing quality. Each parameter carries its own scoring rubric.
Enable, disable, and reweight parameters: Admins can turn parameters on or off and adjust weights directly in the base skill—no cloning required. Weights must sum to 100% across all active parameters.
Custom parameters and categories: Admins can create custom categories and add custom parameters to them directly. Custom parameters cannot be added to base-system categories; for deeper changes to base-system scoring logic, clone the skill through Now Assist Skill Kit.
Manager dashboard: Supervisors see QA scores, category scores, and parameter scores for every agent on their team, with trend analysis and a reviewed-cases list.
Agent dashboard: Live agents see their own QA score and case history, entry point permitting—visibility is gated by a role toggle in the skill's display settings.
Manager override: Managers can edit the AI-generated QA score and individual parameter scores directly on the case page to correct for bias or error.
Mark parameters N/A: Managers can mark a quality score parameter as not applicable on the case detail page when a parameter does not apply to a given case.

Implementation Steps

From Now Assist Admin > Skills, select the Customer workflow and CSM as the product, then activate the Automated quality assurance skill. The guided setup walks you through inputs, scoring parameters and weights, availability, triggers, dashboard access, and role-based display—full steps are in the documentation below.

Best Practices

Pilot before rollout: Test with the skill deactivated for managers and agents first, so you can validate scoring accuracy before rolling it out broadly.
Weights must total 100%: Confirm active parameter weights sum to exactly 100% before activating; the system will not let you save otherwise.
Keep every category covered: Keep at least one parameter active per category to avoid gaps in a pillar's coverage.
Review rubrics with your quality team: Review the rubric text for each parameter with your quality team before enabling it, since the rubric defines how the model scores that parameter.
Use overrides as a signal: Use the manager override sparingly and treat repeated overrides on the same parameter as a signal to revisit its weight or rubric.
Changes are forward-only: Remember that changes to parameters or weights apply to new scores only; historical scores are not recalculated.

Formal Learning

Now Learning: Now Assist for CSM Essentials Path

Measured Success & Outcomes

Outcome	How It Supports Teams	Key Metric
Near-100% QA coverage	Replaces manual sampling (typically 1–2% of cases) with automatic review of every closed case	Percentage of closed cases reviewed
Consistent, defensible scoring	Rubric-based parameters reduce subjectivity between reviewers	Manager override/edit rate
Faster coaching feedback loops	Category and parameter breakdowns show managers exactly where an agent needs support	Time from case close to feedback delivered
Agent self-awareness	Agents can view their own QA score and trends when the role is enabled for the skill	Agent dashboard adoption rate

FAQs

Q. What triggers a quality score to be generated?
A case is scored automatically when it moves to the Closed or Resolved state. No manual action is required.

Q. Can we change which parameters are scored and how much each one counts?
Yes. Admins can enable or disable individual parameters and adjust their weights directly in the base skill, without cloning it. Active parameter weights must sum to 100%.

Q. Can we add our own custom scoring parameters?
Yes, but only within custom categories. Custom parameters cannot be added to the base-system categories; create a custom category first, then add your parameter to it.

Q. Do agents automatically see their own QA scores?
Only if an admin enables the agent role in the skill's display settings. Live agents can only view QA data for cases they personally closed; they cannot see other agents' reviews.

Q. Can a manager correct an AI-generated score?
Yes. Managers can edit the auto-generated QA score and individual parameter scores on the case page, and can mark a parameter as not applicable when it does not apply.

Q. Do changes to parameters affect past scores?
No. Configuration changes apply to new scores only; historical scores remain unchanged.

Q. What if we need deeper customization than the base skill allows?
Clone the skill through Now Assist Skill Kit and ground it with your own evaluation logic and scoring criteria.

Ready to go deeper?

Head back to the Implementation Library Home for the full guided path across ServiceNow Otto for CSM.

Back to Implementation Library Home

JyotiRanjaP

How can we enable the QA Dashboard for a user who can see the score of agents not reporting to him/her?

FernandoCastro

hi @JyotiRanjaP the QA Dashboard for managers is not based out of people hierarchy from AD or else, it is based out of the customer manager role. Once the person has this, they can see all of the contact center agents and slice and dice data as needed.

JyotiRanjaP

Hi @FernandoCastro

Thank you for the response. I checked again and that is not the case on our dashboard. Are we missing any configuration setting on the agent or the dashboard?

This is the situation:

Currently, Some managers are able to see their team scores and also for engineers that report to their reportees.

However, users even having "admin" role along with the customerservice manager and agent roles are not able to see or extract the scores. We are seeing the dashboard at '/now/cwf/agent/autoqa-dashboard/home'.

Kailash Bhange

Hi @JyotiRanjaP

Since Fernando confirmed the manager view is role-based, a hierarchy-like result usually means something else is filtering the data. A few things I'd check:

1. Exact role
Confirm the users have the CSM manager role itself (sn_customerservice_manager), directly or via a group, not only admin. Admin doesn't always pass role checks in UXF dashboards that use specific roles.

2. Skill display settings
In Now Assist Admin > Skills, open the Auto QA / Service Quality skill and check the display/role settings - make sure the manager role is included for the dashboard.

3. Data visibility
Impersonate one of the affected users and open the QA score records in the list view. If they see no rows there, it's an ACL/role (or domain) issue on the score table, not the dashboard. If they see rows, it's the dashboard filter/config.

4. Scores exist
Scores are only generated when cases move to Resolved/Closed with the skill active, so confirm cases for those agents were closed after activation.

5. After role changes, ask the users to log out/in (or clear cache) so the session picks up the new roles.

Could you also share: your Now Assist for Service Quality version, whether domain separation is used, and whether the "working" managers have any extra roles compared to the ones who can't see scores? That comparison usually shows the cause quickly.

Hope this helps - please mark helpful if it does.

AddankiVamK

Hey @JyotiRanjaP ,

One possibility is that the user with manager role is not a member of the assignment group associated with the cases.

To test this;

Take any case that has not been evaluated, assign its assignment_group to the person with the manager role, and then re-trigger AutoQA. This should be reflected in the dashboard.

Nenad Berbatovi

Hi

we had the same problem. Adding the user with the manager role to the assignment_group resolved our issue and manager was able to see QA data