Seeking Real-World ROI and Readiness Assessment Approaches for ServiceNow Otto / Now Assist / AI Age

Jeff Boltz1 · 3 weeks ago

Seeking Real-World ROI and Readiness Assessment Approaches for ServiceNow Otto / Now Assist / AI Agent Advisor

Hello everyone,

We are beginning to evaluate ServiceNow Otto (formerly Now Assist), AI Agent Advisor, and related ITSM AI capabilities and are looking to learn from organizations that have already gone through the process.

Rather than starting with technology features, we're trying to build a business case based on measurable operational outcomes and would appreciate hearing about your experience.

Questions

1. How did you identify the highest-value use cases?

Did you start with:

Incident Management?
Intelligent Incident Routing?
Agent Assist?
Knowledge Recommendations?
Virtual Agent?
Major Incident Management?

What method did you use to prioritize among competing opportunities?

2. How did you establish an ROI baseline?

Before enabling Otto/AI Agent Advisor, did you measure metrics such as:

Average Handle Time (AHT)
MTTR
Reassignment rates
Time spent searching for knowledge
Time spent documenting incidents
First Contact Resolution
Ticket throughput

Were there particular KPIs that proved most useful?

3. How did you assess AI readiness?

What readiness factors mattered most?

For example:

Knowledge article quality and coverage
Incident documentation quality
CMDB maturity
Historical incident volume
Change management processes
User adoption/culture

Did you perform a formal readiness assessment before deployment?

4. What implementation effort was required?

For the use cases that delivered value:

How many FTEs were involved?
What roles participated?
How much effort was platform configuration versus data preparation, knowledge cleanup, governance, and user adoption?

Where did you underestimate effort?

5. Did you perform any controlled testing?

We're considering treating each use case as a hypothesis and validating results against a baseline.

Examples:

Did Agent Assist reduce handling time?
Did Intelligent Routing reduce reassignments?
Did Knowledge Recommendations improve resolution times?

Did anyone use a control group, pilot group, A/B test, or before/after analysis?

6. What surprised you?

Looking back:

Which use cases generated the most value?
Which generated less value than expected?
What prerequisites turned out to be critical?
If you were starting again, what would you do first?

Additional Area of Interest

We're also exploring whether operational telemetry sources (e.g., ServiceNow history, Splunk logs, monitoring data, ticket activity patterns, assignment group metrics, etc.) can be used to identify operational drag before selecting AI use cases.

If you've used data-driven approaches to identify where support organizations spend the most time, I'd be especially interested in hearing about your methodology.

Appreciate any lessons learned, metrics, pitfalls, or success stories you're willing to share.

Thank you!

rpriyadarshy · 3 weeks ago

@Jeff Boltz1 Its a very big post and almost its asking for all in one of Snow Otto 🙂

How did you identify the highest-value use cases? --> Few Parameters and weight which can be considered for this like

Criteria	Weight
Business Impact	30%
Volume	20%
Manual Effort	20%
Ease of Implementation	15%
Data Quality	15%

Use Case	Value	Complexity	Recommendation
Agent Assist	High	Low	Start Day0
Incident Summarization	High	Low	Start Day0
Knowledge Recommendations	High	Low	Start Day0
Intelligent Routing	High	Medium	Day1

Above Table is just a Sample reference

2. How did you establish an ROI baseline?--> Start with OOTB Analytics report and Dashboard like

3. How did you assess AI readiness? What readiness factors mattered most?

You can use Now Assist Center for this. Here is small glimpse of this.

I will try to add response on Other Points Also.

Regards

RP

Jeff Boltz1 · 3 weeks ago

lol - yes a bit much, sorry.

Shorter version:

We are evaluating ServiceNow AI Agent Advisor / Otto and are trying to identify the highest-value use case before pursuing broader adoption.

What was your first use case?
How did you measure ROI?
What prerequisite data quality, knowledge management, or process maturity issues did you discover?

Bonus: Did anyone use ServiceNow metrics, Splunk data, or operational telemetry to identify where support teams spent the most time before selecting AI use cases?

rpriyadarshy · 3 weeks ago

@Jeff Boltz1

Some points which i Picked from all your Posts (here n There 🙂

(1) Virtual Agent for Self Service --> You AI Chat-Bot which is ServiceNow LLM VA can do a quick turnaround for Deflections or Self Service. As this AI Bot will be exposed to whole ORG. One Important Pre-Req is to have a Cleaner KBs . Which KB to expose you can control Using AI Search Profile for VA.

You Can measure deflections also from ServiceNow Table - (sys_cs_deflection_log)

(2) Since Service-Now is the System of record- Manual INC, INC Coming from Monitoring Tools and Other Interfaces. ServiceNow NAC (Now Assist Center) Can also help you to even identify AI Agent/Automation Use Cases. It Uses Clustering mechanism to group.

(3) Measurable Business Value -> For One Customer we did initial maths per use Case.

Let Say UC1 is your Use Case Then Volume/Month and Currently What Time Its taking Per INC.

Calculate Total Time taken and same you do after Implementation. Now for UC1 how Much time you are saving / Month can be converted to $ Value for Business.

Regards

RP

Jeff Boltz1 · 3 weeks ago

Thanks, RP. A couple of themes are becoming clearer from your responses:

Agent Assist, Incident Summarization, and Knowledge Recommendations appear to be emerging as common Day 0 candidates because they are relatively easy to deploy and have measurable outcomes.
Virtual Agent appears highly dependent on KB quality and AI Search maturity. In that sense, it seems as much a Knowledge Management initiative as an AI initiative.
The suggestion to use Now Assist Center and clustering capabilities to identify candidate AI and automation use cases is particularly interesting. We are exploring whether operational data (ServiceNow metrics, Splunk telemetry, assignment patterns, KB usage, etc.) can help identify where support teams spend the most time before prioritizing use cases.

For those who have gone through this exercise, did you identify use cases through a data-driven analysis of operational effort, or did you primarily start with the available AI capabilities and map use cases afterward?