Join the #BuildWithBuildAgent Challenge! Get recognized, earn exclusive swag, and inspire the ServiceNow Community with what you can build using Build Agent.  Join the Challenge.

NOW Assist and AI Search KPI

CA5
Tera Contributor

2 hours ago

What KPIs are used to measure Employee Self-service Enhancement when they start using AI Search on ESC Portal and NOW Assist Gen AI Q & A, Action Genius Result feature?

0 Helpfuls
  • 115 Views
1 REPLY 1

MaxMixali
Tera Guru

an hour ago

Hi, I hope can help: following a list of metrics, grouped by argument that you can use:

 

 

1. Deflection & Resolution Metrics

Case/Ticket Deflection Rate

  • Definition: Percentage of inquiries resolved through self-service without creating a case
  • Formula: (Self-service resolutions / Total inquiries) × 100
  • Target: 30-50% increase after AI implementation
  • Measurement:
    • Track searches that didn't result in case creation
    • Monitor "Did this answer your question?" responses
    • Track virtual agent completions vs. case creations

First Contact Resolution (FCR) Rate

  • Definition: Percentage of issues resolved on first interaction
  • Formula: (Issues resolved on first contact / Total interactions) × 100
  • Target: 60-80%
  • How AI helps: Gen AI Q&A provides accurate answers immediately

Self-Service Resolution Rate

  • Definition: Percentage of cases resolved without agent intervention
  • Measurement: Track cases closed via portal vs. agent-handled
  • Target: 40-60% of total cases

2. Search & Discovery Metrics

Search Success Rate

  • Definition: Percentage of searches that result in a helpful answer
  • Formula: (Successful searches / Total searches) × 100
  • Measurement:
    • Click-through rate on search results
    • Time spent on result pages
    • Positive feedback on results
  • Target: 70-85% success rate

AI Search Accuracy

  • Definition: Relevance of AI-generated responses
  • Measurement:
    • User thumbs up/down on AI answers
    • Follow-up searches (indicates initial failure)
    • Abandon rate after seeing results
  • Target: 80%+ positive feedback

Zero-Result Search Rate

  • Definition: Percentage of searches returning no results
  • Formula: (Searches with no results / Total searches) × 100
  • Target: <5% (should decrease significantly with AI)
  • Baseline vs. AI: Compare traditional search vs. AI search

Action Genius Click-Through Rate

  • Definition: How often users click on suggested actions
  • Formula: (Actions clicked / Actions shown) × 100
  • Target: 40-60%

3. User Engagement Metrics

Portal Adoption Rate

  • Definition: Percentage of employees using ESC portal
  • Formula: (Active portal users / Total employees) × 100
  • Target: 70-90% adoption
  • Track: Monthly active users (MAU)

Average Time on Portal

  • Definition: Average session duration
  • Measurement: Time from login to logout
  • Expected change: May decrease (faster answers) or increase (more engagement)

Repeat Usage Rate

  • Definition: Users returning to self-service
  • Formula: (Returning users / Total users) × 100
  • Target: 60%+ return rate

Feature Utilization Rate

  • AI Search usage: % of employees using AI search vs. traditional search
  • Gen AI Q&A usage: Number of AI-generated answers requested
  • Action Genius usage: Number of suggested actions taken
  • Target: 50%+ adoption of AI features within 6 months

4. Efficiency & Speed Metrics

Average Time to Resolution (Self-Service)

  • Definition: Time from search to resolution
  • Measurement: Time stamp analysis
  • Target: 50-70% reduction with AI
  • Baseline: 10-15 minutes → Target: 3-5 minutes

Time to First Answer

  • Definition: How quickly AI provides a response
  • Target: <5 seconds for Gen AI Q&A
  • Measurement: System response time logs

Mean Time Between Requests (MTBR)

  • Definition: Time between employee requests
  • Expected change: Should increase (fewer repeat questions)
  • Indicates: Better first-time resolution

Search Refinement Rate

  • Definition: Number of follow-up searches needed
  • Formula: (Refined searches / Initial searches) × 100
  • Target: <20% (indicates first search was successful)

5. Cost & Productivity Metrics

Cost Per Ticket (Self-Service vs. Agent)

  • Self-service cost: $1-3 per resolution
  • Agent-handled cost: $15-30 per resolution
  • ROI calculation: Savings from deflection × deflection rate

Agent Productivity

  • Average Handle Time (AHT): Should decrease as simple queries are deflected
  • Cases per Agent: May increase as agents handle complex issues only
  • Agent Utilization Rate: More time on value-added work

HR/IT Support Cost Savings

  • Formula: (Deflected tickets × Cost per agent ticket) - AI implementation cost
  • Target: 20-40% cost reduction
  • Example: 1,000 deflected tickets/month × $20 = $20,000 monthly savings

6. User Satisfaction Metrics

Customer Satisfaction Score (CSAT)

  • Definition: User satisfaction with self-service experience
  • Measurement: Post-interaction survey (1-5 scale)
  • Question: "How satisfied were you with the self-service experience?"
  • Target: 4.0+ / 5.0

Net Promoter Score (NPS)

  • Question: "How likely are you to recommend the ESC portal to colleagues?"
  • Scale: 0-10
  • Target: 50+ NPS

Effort Score (CES)

  • Question: "How easy was it to find what you needed?"
  • Scale: 1-7 (1 = Very Difficult, 7 = Very Easy)
  • Target: 6+ / 7

AI Answer Quality Rating

  • Thumbs up/down on AI responses
  • Formula: (Thumbs up / Total ratings) × 100
  • Target: 80%+ positive ratings

Feedback Sentiment Analysis

  • Analyze: Text feedback on AI answers
  • Tool: Sentiment scoring (Positive/Neutral/Negative)
  • Target: 70%+ positive sentiment

 

0 Helpfuls