When to Use AI Agents: Rationalizing Uses Cases for Workflows, GenAI Skills & AI Agents

[TJL]

Introduction

Welcome back to our series of articles from the AI Center of Excellence (CoE) team at ServiceNow.  We are a group of dedicated AI Strategists and Architects committed to advancing the implementation and adoption of AI solutions for our customers. Through countless advisory and hands-on engagements, we’ve gathered valuable insights and practical guidance that we’re excited to share with the broader ServiceNow community. During recent customer conversations architectural tradeoffs and considerations for when to use a workflow versus augmenting a workflow with Generative AI (GenAI) capabilities versus leveraging AI Agents has initiated vigorous discussion. 

This article outlines: 

• The GenAI architectural patterns which can augment workflows on ServiceNow 
• Design tradeoffs between using a workflow, GenAI Skills or leveraging AI Agents and where they fall on a spectrum of deterministic to probabilistic 
• Provides insights into different use cases, regulatory compliance concerns, cost and scalability considerations, and data dependencies for each pattern 

GenAI Experiences on ServiceNow 

Before discussing the tradeoffs, let’s define the architectural patterns where GenAI can augment workflows and how they apply on ServiceNow: 

1.   Workflow:  

A workflow is a script or flow which is deterministic. The input to a workflow is well-defined, and the logic is rule-based (e.g., if-then statements) making it the most deterministic option available to ServiceNow architects. 

ServiceNow Capability:  

Workflow Subflow or Action or a Script 

Example:  

A workflow can be short and simple like auto-assigning an incident to a queue when a specific category is selected on a record or more complex and long-running like automating the employee onboarding experience 

2.   GenAI Skills: 

Workflows and experiences can be enhanced with GenAI skills to improve fulfiller productivity and improve requester self-service. Now Assist skills are GenAI-powered use cases that are integrated into workflows. GenAI is well-suited for a variety of use cases including summarization, question answering, information extraction, classification and content generation. Skills leverage tools to bring in both structured (record) and unstructured (the body of knowledge articles) data into a prompt template at runtime. This pattern is going to be slightly less deterministic than a rule-based workflow. However, skills typically have a human-in-the-loop and can be made more predictable by following strong prompt engineering principles and grounding the prompt in the necessary context from a customer’s instance.  

ServiceNow Capability: 

There are a variety of skills which customers will have access to out-of-the-box within the Now Assist portfolio for modules like ITSM, CSM or HRSD but custom skills can be created using Now Assist Skill Kit. 

Example:  

ServiceNow skills are invoked when a user performs certain actions. 

A few examples of the out-of-the-box skills which can augment the fulfiller workflow are: 

• Clicking the Summarize button (UI Action) to Summarize a record 
• Clicking the Now Assist sparkle in the Resolution Notes field to generate a Resolution Note 
• Using the Now Assist Panel to invoke an incident assistant which finds related incidents 

If there’s an interest in creating a custom skill, the documented procedure for creating a Knowledge Article Coach can be created in Now Assist Skill Kit and is a sought-after example many customers find compelling. 

3.   Prompt Chaining: 

Prompt chaining breaks down complex tasks into a sequence of smaller skills by using the results from previous skills as context for the next skill in the chain. Notwithstanding architectures which poses advanced reasoning capabilities, each prompt should focus on one task that GenAI does well. By chaining prompts and skills together, you can create purpose-built prompts for tasks like analysis, classification, summarization and email writing. Prompt chaining holds promise as a pattern which can be more sophisticated than a single GenAI skill but more deterministic than Agentic AI and can be used to augment each architecture. 

ServiceNow Capability:  

Flow actions and Now Assist Skill Kit allow you to chain prompts together. 

Example: 

When supporting customers, representatives will typically analyze sentiment and adjust their messages accordingly. GenAI can do the same.  The first skill in the chain can analyze the sentiment of an inbound email. Based upon the sentiment, a second skill can be created for positive sentiment or for neutral sentiment which identifies the best tone for the response (e.g., friendly, empethetic, serious, professional). A third skill can be invoked in the chain which drafts a personalized email using the original email, the identified sentiment and tone modifications as context. Notice how negative sentiment was omitted. Inbound emails with negative sentiment can be categorized and then handled by a human, or responses can be generated but require agents to review and approve before sending. Classifying the sentiment as step one, delivers the benefits of a rule-based system – if positive/neutral do X but if negative do Y – but reduces the risk of further upsetting the customer, ensuring customer satisfaction is maintained. 

4.   Agentic AI:  

The primary pattern with Agentic AI on ServiceNow is a strategy named ReAct or reasoning and action. The AI agent can reason through the request, analyze the tools which it can use, create a plan, and execute the plan by calling those tools. After each tool invocation, it can modify its plan and repeat until the agent reaches the desired outcome. However, the action portion of the ReAct strategy is truly differentiated. Instead of just responding to a user, as if it’s a chatbot (which is also a capability) it can take sophisticated actions like sending emails or invoking create and update operations on table records. The input to AI Agents can be vague. The context, which can be a combination of structured record data, semi-structured data or unstructured data (like knowledge or attachments), is gathered through tool use at run-time to create the desired response. AI Agents can be either fully autonomous, which takes actions on the user’s behalf, or supervised, which will ask for the user’s permission before taking certain actions. Architects should lean towards supervised when they wish to keep a human fulfiller in the decision loop, but autonomous often drives the most productivity savings with event-driven invocation. 

ServcieNow Capability: 

AI Agents are created in AI Agent Studio in ServiceNow where you can build and manage Agentic workflows and AI Agents. AI Agents can be reusable across workflows. For instance, developers might create an AI Agent library and have one AI Agent which writes emails and a second which extracts text from PDF email attachments using Now Assist for Document Intelligence. Those two AI Agents could be used in Agentic Workflows for both triaging cases within the CSM module or assisting with incidents in ITSM use cases. The Agentic Workflow uses an orchestrator to delegate work to its fleet of AI Agents. Similarly, the AI agents delegate work to its tools to gather data, manipulate data or take other actions. It’s important to note that Workflow Studio and Virtual Agent Studio have integration points into AI Agents but not Agentic Workflows. 

Example:  

ServiceNow ships a variety of out-of-the-box agentic workflows (which include subordinate AI Agents) for use cases such as triaging Customer Service Cases, Generating Resolution Plans or Resolving Non-Critical HR Cases. 

A Spectrum of Deterministic to More Probabilistic 

Now that a firm understanding of the different GenAI patterns has been discussed, let’s summarize on a spectrum of deterministic/less probabilistic to probabilistic/less deterministic: 

• Workflow (Deterministic) 
• GenAI Skills (Less Deterministic, slightly probabilistic with a human-in the loop) 
• Agentic AI (Least deterministic and most probabilistic with options for autonomous or supervised action taking) 

Notice how prompt chaining is missing from the spectrum above. Prompt chaining enhances GenAI Skills and Agentic AI as a method to ground the workflow and make each pattern more deterministic – more on this later. 

Use Case-Based Architectural Tradeoffs  

Customers often ask under what circumstances should you use one pattern versus another, so we’ll analyze that across three factors: 

1.   Regulatory Compliance & Threshold for Risk:  

Transitioning from a deterministic system, where users expect predictability, to a probabilistic workflow can be challenging for organizations with low risk tolerance.  These risks come to the forefront in highly regulated environments. Imagine that you have a use case which processes loan documentation in banking or understands clinical notes in healthcare. In either of those two use cases, GenAI hallucinations may be unacceptable. Furthermore, regulations may require that you maintain explainability and, while logs can be surfaced for each pattern, they may not be readily understood by non-technical audiences like auditors. Due to these factors, many customers with a low risk tolerance or in regulated industries favor the deterministic end of the spectrum. When these customers decide to integrate GenAI capabilities into their workflow, many will maintain a preference for workflows where human reviewers can oversee the output and make key decisions. 

Summary:  

Use Workflows in regulatory environments or in use cases where minimizing risk is paramount. Consider GenAI skills to augment human knowledge workers which maintains a human-in-the-loop. The decision logs in Agentic AI may not be easily understood by regulators or auditors. While Now Assist Guardian can filter out offensive content and guard against prompt manipulation, it doesn't allow for custom guardrails. Some common examples of custom guardrails include blocking responses which might be considered financial or healthcare advice. For these types of use cases, explore a prompt-chain during either pre- or post-processing to rewrite or block the query and response. 

2.   Cost & Scalability:  

The spectrum of deterministic and probabilistic aligns well to cost. Generally, cost increases as the workflow or experience becomes more probabilistic. The unit of measurement for GenAI skills on ServiceNow is called Assists. In contrast to token-based metering, an Assist is based upon the value and complexity of the GenAI capability. To provide customers with intuition on how many Assists they can expect to consume, ServiceNow publishes a Now Assist Rate Card. By using the rate card customers can estimate their Assist consumption by skill.  

On the deterministic end of the spectrum, workflows aren’t injected with GenAI capabilities, so no Assists are consumed. However, when you start infusing workflows and experiences with GenAI features, use the Now Assist Rate Card for Assist forecasting. For five example skills, Assist consumption is as follows: 

• Custom Skill in Now Assist Skill Kit = 1 Assist 
• Record Summarization = 1 Assist  
• Q&A Genius Results = 1 Assist 
• End-to-End LLM Topic Invocation = 10 Assists   
• Agentic Workflow (Small, 0-4 actions) = 25 Assists 

Disclaimer: The Assist consumption data was taken from the Now Assist Rate Card at the time this article was published. Please refer to the original document for the latest information. 

Consider a use case where a user is having a conversation with Virtual Agent and asks about a company policy and then requests that an email be sent to a user with the details.  

An AI Agent workflow might look something like this: 

• Q&A Genius Results to find the policy in Knowledge Management = 1 Assist  
• AI Agent (small workflow) to write and send an email = 25 Assists 

Total: 26 Assists 

A functionally equivalent but more cost-effective solution (consuming 53% less Assists) may be designed as follows: 

• Q&A Genius Results to find the policy in Knowledge Management = 1 Assist  
• Custom Skill in Now Assist Skill Kit to write an email = 1 Assist  
• LLM Topic Invocation to call the custom skill and send email = 10 Assists  

Total: 12 Assists 

This is just one simplistic example which is purely illustrative. The thought exercise is designed to demonstrate how to think differently about your architecture to limit assist consumption. It’s also worthwhile to map out AI Agent tool usage and consider which series of tools can be consolidated into one tool consisting of a longer workflow, a skill or prompt chain to limit actions.  

The number of actions used by the AI Agent at runtime can take your AI Agent from one size to the next tier: 

• Small (0-4 actions) consuming 25 Assists  
• Medium (5-8 actions) consuming 50 Assists  
• Large (9-20 actions) consuming 150 Assists  

Being thoughtful about assigning the right tools, or a longer workflow to your AI Agent can prevent unnecessary action expansion, potentially pushing the AI Agent to the next size. When tools don’t need to be invoked independently, longer workflows allow you to effectively inject the output from more deterministic patterns into your AI Agent. 

Summary:  

Combine workflows, GenAI skills and prompt chains into your AI Agents to add more deterministic patterns to your AI Agent and curtail Assist consumption. When deploying your AI Agent, get a minimally viable version working and then eliminate technical debt iteratively by consolidating tools together when possible. Consider alternative architectures which can achieve the same outcome with greater value. When an alternative GenAI pattern can suffice consider GenAI skills or prompt chaining to save on Assists.  

3. Use Case, Data and Dependencies:  

Earlier some of the data considerations for each pattern were discussed. To recap, generally if the data is structured or semi-structured it’ll align well to the more deterministic end of the spectrum. Conversely if the data is vague, unstructured (body of a knowledge article or web search), needs to leverage Now Assist in AI Search, or needs to reason through the ask then it’ll be a great use case for AI Agents.  

Interview knowledge workers to determine where they’re making decisions during their workflow or whether they sometimes perform steps in slightly different orders. If the answer is yes, then determine whether exposing that data to an AI Agent is feasible for it to make a similar judgement. If a user invokes GenAI capabilities through a UI Action, an on-screen modal, through the Now Assist Panel or Virtual Agent to make them more productive then GenAI skills are likely the place to start. Use cases like information summarization, creating a first draft of net-new content, editing existing content or information retrieval are all great options for GenAI-powered skills. Some of these skills may get used in your AI Agents in the fullness of time so investing time here first can often be viewed as foundational work to make AI Agents successful. If you can script an outcome or build it in workflow studio without needing to call GenAI capabilities, build those workflows out and use them in your GenAI skill, prompt chain or in your AI Agents.  

Summary: 

Anthropomorphize your AI Agent (give it human-like features) to determine whether it can take over the decision making and reasoning steps which knowledge workers are performing today. Next, determine if the data is available or can be available on ServiceNow. For other home game capabilities in GenAI, default to GenAI skills or prompt chaining to surface information to fulfillers and requesters over the channels which they currently use.  

Conclusion:

Discussed were the top three decision points when choosing between a workflow, GenAI skills or AI Agents however, it’s not exhaustive. As the prevalence of AI agents increases, scalability and performance of the platform will need to be considered. Factors like latency and wait time for end-users should be considered to ensure that a positive user-experience is maintained, and organizational change management (OCM) ought to be planned proactively. These topics will be discussed in a dedicated article as a continuation of this topic.

Choosing the right AI pattern is critical for maximizing value and benefit. By considering factors like regulatory compliance, assist consumption, use cases, data and platform dependencies you can ensure your GenAI solution on ServiceNow enhances productivity and automates workflows at the lowest cost.  

If you have questions or thoughts, feel free to drop them in the comments—we’ll respond or update the article as needed. If you found this article helpful, please share your feedback or link to it on your preferred platform. This is just the beginning of our series on AI – stay tuned for more! 

For tailored guidance, reach out to your ServiceNow account team.   

𝘗𝘚: 𝘝𝘪𝘦𝘸𝘴 𝘢𝘳𝘦 𝘮𝘺 𝘰𝘸𝘯, 𝘢𝘯𝘥 𝘥𝘰 𝘯𝘰𝘵 𝘳𝘦𝘱𝘳𝘦𝘴𝘦𝘯𝘵 𝘮𝘺 𝘵𝘦𝘢𝘮, 𝘦𝘮𝘱𝘭𝘰𝘺𝘦𝘳, 𝘱𝘢𝘳𝘵𝘯𝘦𝘳𝘴, 𝘰𝘳 𝘤𝘶𝘴𝘵𝘰𝘮𝘦𝘳𝘴.