Seeking Real-World ROI and Readiness Assessment Approaches for ServiceNow Otto / Now Assist / AI Age

[Jeff Boltz1]

Seeking Real-World ROI and Readiness Assessment Approaches for ServiceNow Otto / Now Assist / AI Agent Advisor

Hello everyone,

We are beginning to evaluate ServiceNow Otto (formerly Now Assist), AI Agent Advisor, and related ITSM AI capabilities and are looking to learn from organizations that have already gone through the process.

Rather than starting with technology features, we're trying to build a business case based on measurable operational outcomes and would appreciate hearing about your experience.

Questions

1. How did you identify the highest-value use cases?

Did you start with:

• Incident Management?
• Intelligent Incident Routing?
• Agent Assist?
• Knowledge Recommendations?
• Virtual Agent?
• Major Incident Management?

What method did you use to prioritize among competing opportunities?

---

2. How did you establish an ROI baseline?

Before enabling Otto/AI Agent Advisor, did you measure metrics such as:

• Average Handle Time (AHT)
• MTTR
• Reassignment rates
• Time spent searching for knowledge
• Time spent documenting incidents
• First Contact Resolution
• Ticket throughput

Were there particular KPIs that proved most useful?

---

3. How did you assess AI readiness?

What readiness factors mattered most?

For example:

• Knowledge article quality and coverage
• Incident documentation quality
• CMDB maturity
• Historical incident volume
• Change management processes
• User adoption/culture

Did you perform a formal readiness assessment before deployment?

---

4. What implementation effort was required?

For the use cases that delivered value:

• How many FTEs were involved?
• What roles participated?
• How much effort was platform configuration versus data preparation, knowledge cleanup, governance, and user adoption?

Where did you underestimate effort?

---

5. Did you perform any controlled testing?

We're considering treating each use case as a hypothesis and validating results against a baseline.

Examples:

• Did Agent Assist reduce handling time?
• Did Intelligent Routing reduce reassignments?
• Did Knowledge Recommendations improve resolution times?

Did anyone use a control group, pilot group, A/B test, or before/after analysis?

---

6. What surprised you?

Looking back:

• Which use cases generated the most value?
• Which generated less value than expected?
• What prerequisites turned out to be critical?
• If you were starting again, what would you do first?

---

Additional Area of Interest

We're also exploring whether operational telemetry sources (e.g., ServiceNow history, Splunk logs, monitoring data, ticket activity patterns, assignment group metrics, etc.) can be used to identify operational drag before selecting AI use cases.

If you've used data-driven approaches to identify where support organizations spend the most time, I'd be especially interested in hearing about your methodology.

Appreciate any lessons learned, metrics, pitfalls, or success stories you're willing to share.

Thank you!

[Jeff Boltz1]

Thanks, RP. I really like the weighted scoring approach and think it provides a practical framework for evaluating competing use cases.

I made one small addition based on discussions we're having internally:

• Business Impact (25%)
• Volume (20%)
• Manual Effort (15%)
• Ease of Implementation (15%)
• Data Quality (15%)
• Adoption Likelihood (10%)

My thinking is that even a high-value use case may struggle to deliver ROI if agents don't trust it or incorporate it into their daily workflow.

Using that framework, I could see an initial prioritization looking something like:

Day 0

• Agent Assist
• Incident Summarization
• Knowledge Recommendations

Day 1

• Intelligent Routing

One area I'm particularly interested in learning from customers is how they moved from prioritization to measurable business value.

For organizations that have deployed these capabilities:

• What KPIs did you baseline before implementation?
• What outcomes improved the most (MTTR, handling time, reassignment rates, KB utilization, ticket deflection, etc.)?
• How much effort was platform configuration versus knowledge cleanup, data quality improvements, governance, and user adoption?
• Were there any readiness indicators that accurately predicted success or failure?

We're also exploring whether ServiceNow operational metrics, Splunk telemetry, assignment group data, and incident activity patterns can help identify the biggest sources of operational effort ("operational drag") before selecting AI use cases.

Has anyone used ServiceNow operational data, monitoring data, or log analytics to identify where support teams spend the most time before prioritizing AI investments? If so, what methodology did you use and what insights emerged?

Thanks again for sharing your experience. It definitely helps move the discussion from "what AI can do" to "where AI can create measurable value."

P.S. During discussions with ServiceNow advisors, several additional candidate use cases were suggested:

• Virtual Agent for Self-Service
• Predictive Major Incident Identification
• Proactive Problem Management
• Automated Change Risk Assessment
• Dynamic Translation for Global Support
• Sentiment Analysis
• Request Fulfillment Automation
• Performance Anomaly Detection
• IT Asset Lifecycle Optimization
• Automated Service Outage Notification
• Predictive User Experience Monitoring
• Automated Incident Resolution for Known Errors

For organizations that have implemented Otto / Now Assist / AI Agent Advisor, which of these delivered the fastest measurable ROI, and which required more organizational maturity (knowledge management, CMDB quality, process maturity, historical data quality, etc.) than initially expected?