NOW Assist and AI Search KPI

[CA5]

What KPIs are used to measure Employee Self-service Enhancement when they start using AI Search on ESC Portal and NOW Assist Gen AI Q & A, Action Genius Result feature?

[MaxMixali]

Hi, I hope can help: following a list of metrics, grouped by argument that you can use:

1. Deflection & Resolution Metrics

Case/Ticket Deflection Rate

• Definition: Percentage of inquiries resolved through self-service without creating a case
• Formula: (Self-service resolutions / Total inquiries) × 100
• Target: 30-50% increase after AI implementation
• Measurement:
  • Track searches that didn't result in case creation
  • Monitor "Did this answer your question?" responses
  • Track virtual agent completions vs. case creations

First Contact Resolution (FCR) Rate

• Definition: Percentage of issues resolved on first interaction
• Formula: (Issues resolved on first contact / Total interactions) × 100
• Target: 60-80%
• How AI helps: Gen AI Q&A provides accurate answers immediately

Self-Service Resolution Rate

• Definition: Percentage of cases resolved without agent intervention
• Measurement: Track cases closed via portal vs. agent-handled
• Target: 40-60% of total cases

2. Search & Discovery Metrics

Search Success Rate

• Definition: Percentage of searches that result in a helpful answer
• Formula: (Successful searches / Total searches) × 100
• Measurement:
  • Click-through rate on search results
  • Time spent on result pages
  • Positive feedback on results
• Target: 70-85% success rate

AI Search Accuracy

• Definition: Relevance of AI-generated responses
• Measurement:
  • User thumbs up/down on AI answers
  • Follow-up searches (indicates initial failure)
  • Abandon rate after seeing results
• Target: 80%+ positive feedback

Zero-Result Search Rate

• Definition: Percentage of searches returning no results
• Formula: (Searches with no results / Total searches) × 100
• Target: <5% (should decrease significantly with AI)
• Baseline vs. AI: Compare traditional search vs. AI search

Action Genius Click-Through Rate

• Definition: How often users click on suggested actions
• Formula: (Actions clicked / Actions shown) × 100
• Target: 40-60%

3. User Engagement Metrics

Portal Adoption Rate

• Definition: Percentage of employees using ESC portal
• Formula: (Active portal users / Total employees) × 100
• Target: 70-90% adoption
• Track: Monthly active users (MAU)

Average Time on Portal

• Definition: Average session duration
• Measurement: Time from login to logout
• Expected change: May decrease (faster answers) or increase (more engagement)

Repeat Usage Rate

• Definition: Users returning to self-service
• Formula: (Returning users / Total users) × 100
• Target: 60%+ return rate

Feature Utilization Rate

• AI Search usage: % of employees using AI search vs. traditional search
• Gen AI Q&A usage: Number of AI-generated answers requested
• Action Genius usage: Number of suggested actions taken
• Target: 50%+ adoption of AI features within 6 months

4. Efficiency & Speed Metrics

Average Time to Resolution (Self-Service)

• Definition: Time from search to resolution
• Measurement: Time stamp analysis
• Target: 50-70% reduction with AI
• Baseline: 10-15 minutes → Target: 3-5 minutes

Time to First Answer

• Definition: How quickly AI provides a response
• Target: <5 seconds for Gen AI Q&A
• Measurement: System response time logs

Mean Time Between Requests (MTBR)

• Definition: Time between employee requests
• Expected change: Should increase (fewer repeat questions)
• Indicates: Better first-time resolution

Search Refinement Rate

• Definition: Number of follow-up searches needed
• Formula: (Refined searches / Initial searches) × 100
• Target: <20% (indicates first search was successful)

5. Cost & Productivity Metrics

Cost Per Ticket (Self-Service vs. Agent)

• Self-service cost: $1-3 per resolution
• Agent-handled cost: $15-30 per resolution
• ROI calculation: Savings from deflection × deflection rate

Agent Productivity

• Average Handle Time (AHT): Should decrease as simple queries are deflected
• Cases per Agent: May increase as agents handle complex issues only
• Agent Utilization Rate: More time on value-added work

HR/IT Support Cost Savings

• Formula: (Deflected tickets × Cost per agent ticket) - AI implementation cost
• Target: 20-40% cost reduction
• Example: 1,000 deflected tickets/month × $20 = $20,000 monthly savings

6. User Satisfaction Metrics

Customer Satisfaction Score (CSAT)

• Definition: User satisfaction with self-service experience
• Measurement: Post-interaction survey (1-5 scale)
• Question: "How satisfied were you with the self-service experience?"
• Target: 4.0+ / 5.0

Net Promoter Score (NPS)

• Question: "How likely are you to recommend the ESC portal to colleagues?"
• Scale: 0-10
• Target: 50+ NPS

Effort Score (CES)

• Question: "How easy was it to find what you needed?"
• Scale: 1-7 (1 = Very Difficult, 7 = Very Easy)
• Target: 6+ / 7

AI Answer Quality Rating

• Thumbs up/down on AI responses
• Formula: (Thumbs up / Total ratings) × 100
• Target: 80%+ positive ratings

Feedback Sentiment Analysis

• Analyze: Text feedback on AI answers
• Tool: Sentiment scoring (Positive/Neutral/Negative)
• Target: 70%+ positive sentiment

[Simon Hendery]

Dude, let's not do this. #AI_slop