ServiceNow Otto for CSM - Automated Quality Assurance

FernandoCastro · 2 hours ago

Automatically score and coach every closed case with GenAI-generated quality reviews

Automated quality assurance reviews closed cases against a configurable rubric and generates a quality score with qualitative feedback for each one, replacing manual spot-checks with consistent, near-100% coverage. Managers and agents see the results in role-specific dashboards, and admins control which parameters count and how much each one is weighted.

Automated quality assurance is delivered by the Now Assist for Service Quality v1.0 app, which is auto-installed as part of Now Assist for CSM. You will not need to separately activate it, but it may appear under its own app name in Studio or the application list.

Key Capabilities

Automatic review on case close: The skill triggers when a case moves to Closed or Resolved, so every case gets scored without agents or managers doing anything.
Configurable rubric library: Nine base-system categories and 21 parameters cover issue understanding, communication quality, empathy, knowledge base use, first-contact resolution, and closing quality. Each parameter carries its own scoring rubric.
Enable, disable, and reweight parameters: Admins can turn parameters on or off and adjust weights directly in the base skill—no cloning required. Weights must sum to 100% across all active parameters.
Custom parameters and categories: Admins can create custom categories and add custom parameters to them directly. Custom parameters cannot be added to base-system categories; for deeper changes to base-system scoring logic, clone the skill through Now Assist Skill Kit.
Manager dashboard: Supervisors see QA scores, category scores, and parameter scores for every agent on their team, with trend analysis and a reviewed-cases list.
Agent dashboard: Live agents see their own QA score and case history, entry point permitting—visibility is gated by a role toggle in the skill's display settings.
Manager override: Managers can edit the AI-generated QA score and individual parameter scores directly on the case page to correct for bias or error.
Mark parameters N/A: Managers can mark a quality score parameter as not applicable on the case detail page when a parameter does not apply to a given case.

Implementation Steps

From Now Assist Admin > Skills, select the Customer workflow and CSM as the product, then activate the Automated quality assurance skill. The guided setup walks you through inputs, scoring parameters and weights, availability, triggers, dashboard access, and role-based display—full steps are in the documentation below.

Best Practices

Pilot before rollout: Test with the skill deactivated for managers and agents first, so you can validate scoring accuracy before rolling it out broadly.
Weights must total 100%: Confirm active parameter weights sum to exactly 100% before activating; the system will not let you save otherwise.
Keep every category covered: Keep at least one parameter active per category to avoid gaps in a pillar's coverage.
Review rubrics with your quality team: Review the rubric text for each parameter with your quality team before enabling it, since the rubric defines how the model scores that parameter.
Use overrides as a signal: Use the manager override sparingly and treat repeated overrides on the same parameter as a signal to revisit its weight or rubric.
Changes are forward-only: Remember that changes to parameters or weights apply to new scores only; historical scores are not recalculated.

Formal Learning

Now Learning: Now Assist for CSM Essentials Path

Measured Success & Outcomes

Outcome	How It Supports Teams	Key Metric
Near-100% QA coverage	Replaces manual sampling (typically 1–2% of cases) with automatic review of every closed case	Percentage of closed cases reviewed
Consistent, defensible scoring	Rubric-based parameters reduce subjectivity between reviewers	Manager override/edit rate
Faster coaching feedback loops	Category and parameter breakdowns show managers exactly where an agent needs support	Time from case close to feedback delivered
Agent self-awareness	Agents can view their own QA score and trends when the role is enabled for the skill	Agent dashboard adoption rate

FAQs

Q. What triggers a quality score to be generated?
A case is scored automatically when it moves to the Closed or Resolved state. No manual action is required.

Q. Can we change which parameters are scored and how much each one counts?
Yes. Admins can enable or disable individual parameters and adjust their weights directly in the base skill, without cloning it. Active parameter weights must sum to 100%.

Q. Can we add our own custom scoring parameters?
Yes, but only within custom categories. Custom parameters cannot be added to the base-system categories; create a custom category first, then add your parameter to it.

Q. Do agents automatically see their own QA scores?
Only if an admin enables the agent role in the skill's display settings. Live agents can only view QA data for cases they personally closed; they cannot see other agents' reviews.

Q. Can a manager correct an AI-generated score?
Yes. Managers can edit the auto-generated QA score and individual parameter scores on the case page, and can mark a parameter as not applicable when it does not apply.

Q. Do changes to parameters affect past scores?
No. Configuration changes apply to new scores only; historical scores remain unchanged.

Q. What if we need deeper customization than the base skill allows?
Clone the skill through Now Assist Skill Kit and ground it with your own evaluation logic and scoring criteria.

Ready to go deeper?

Head back to the Implementation Library Home for the full guided path across ServiceNow Otto for CSM.

Back to Implementation Library Home