From Development to Production: Ask ServiceNow's AI Experts Anything

[ryanyong]

Hey everyone, I'm Ryan, an engineer on ServiceNow's Forward Deployed Engineering team. We're a team of experts and builders who've gained deep AI deployment expertise at tier 1 technology companies and research labs. We're certified experts on the ServiceNow AI Platform, with direct access to Product and the ServiceNow C-suite.

What we do
We partner directly with customers facing the most complex challenges to co-innovate agentic AI use cases and ship them to production on the ServiceNow AI Platform.

Why this thread
If you're a ServiceNow customer sitting on AI entitlement you haven't fully put to work, this is your chance to ask the Forward Deployed Engineers building these solutions in the field. Bring your hardest questions about moving from unused credits to AI running in production, and getting the most value out of your existing ServiceNow investment.

Ask us anything. We are the subject matter experts, and we will get you an answer.

We'll be answering live from August 3rd to 7th. Post your question anytime between now and 8/7, upvote the ones you want answered first.

[Dustin Gannon]

This is a good question, and it's actually one I've had to work through with a customer in Germany recently.

The challenge is real because we're innovating on the platform so quickly that there are several different layers where AI agents and their logic can run. Figuring out perfect reusability is almost a second-order optimization while the platform itself is still evolving very quickly.

That said, there are some common-sense practices that help a lot.

If I'm writing a tool for an AI agent and I know that business logic needs to be used by multiple agents, I generally put that logic into a Script Include and have a much smaller piece of agent-specific code call it.

That means the underlying function or collection of business logic can be reused between voice agents, chat agents, and potentially other experiences.

This also aligns with general ServiceNow development and test-automation best practices. Even though the toolsets allow inline scripting, I almost always prefer putting substantial business logic into Script Includes and calling those from the inline scripting layer rather than putting large scripts directly inside the AI configuration.

That improves reusability, maintainability, testability, and production stability.

I do agree with the premise of the question, though: this is an area where the products themselves can continue becoming better aligned. It still feels a little like the Wild West.

We can achieve almost anything we want today, but depending on the use case, you may still find yourself with some duplicate logic or even versions of the same tool being built more than once.

[Marc_007]

Hello SNOW Gurus,

     We are looking into Agentic Agents for the SOC. What are the most common pit falls and challenges for your customer during the planning and testing phases? Thanks

[Dustin Gannon]

Assuming SOC here means Security Operations Center, the first thing I would focus on is your testing strategy.

You need to know how you're going to test the solution before you build it, because your testing strategy is what ultimately tells you whether you've reached your definition of done.

Think about the experimentation framework, the testing harnesses, and how you're going to measure success from the very beginning.

I spend as much time thinking about the happy path as I do about corner cases and edge cases.

As a rough rule, I might make 60–70% of the evaluation dataset representative happy-path scenarios.

Then take some of those happy paths, find their inverse, and deliberately build negative tests. I might dedicate roughly 10% of the testing plan to that.

Another 10–15% should be specifically shaped around the business problem—the unusual situations, important goals, or failure modes that are particularly relevant to that customer.

The broader point is: figure out your eval strategy early.

You can use ServiceNow's evaluation frameworks, Data Kit, AI Skill Kit, and other platform capabilities. You can also use newer Fluent capabilities to vibe-code an evaluation set when you have more specialized requirements.

One technique I've found particularly interesting is evaluating the entire agentic journey rather than evaluating only individual tools, calls, or functions.

Take the entire agentic execution, put it through an evaluator, define the business outcome you're trying to achieve, and ask an LLM to judge whether that execution actually accomplished the outcome.

That's essentially an LLM-as-a-judge approach.

Now you can run that across 100 or 1,000 executions and start answering a much more valuable question at a data-driven level: Are we actually achieving the business outcome?

It doesn't replace human thinking or human reasoning. But it gives you the ability to reason across large datasets and develop a general signal for whether one version, prompt, or design is better than the previous one.

For me, “How am I going to test this?” is usually one of the very first questions in the design process.

If I'm trying to achieve something, I need to know how I'm going to measure it right off the bat.

[gulatia]

We are trying to roll out Now Assist with Co-pilot.  We followed steps documented on Microsoft Copilot integration with Now Assist FAQ ... - ServiceNow Community

We are running into an issue where with the Now Assist Co-pilot integration, we have to refresh the co-pilot window for the output from ServiceNow Agent to show up. The same functionality works without any issues if we are in the BOT itself as compared to co-pilot. Based on the investigation so far, we do see the Conversation messages in ServiceNow but are not able to view those in Co-pilot unless we refresh the UI.

We are currently working with SN Support and Microsoft support but have no resolution so far. Any recommendations on how to resolve this?

Additionally, i am trying to understand how this functionality differs with and without Now Assist enabled in ServiceNow.  I am trying to check if the messages from BOT are using Now assist or not.

Thanks

Aman

[Dustin Gannon]

This is a very implementation-specific question, and based only on the outside-in description, I wouldn't want to pretend I can diagnose the exact issue.

If the conversation messages are successfully appearing in ServiceNow but aren't rendering in the Copilot or Teams experience until the UI refreshes, that certainly points toward investigating the integration and rendering side of the experience.

Given that you're already working with both ServiceNow Support and Microsoft Support, my recommendation would be to get forward-deployed technical experts from both companies into the same room at the same time.

That's often the fastest way to stop a problem from bouncing back and forth across organizational boundaries.

It may not be instantly solvable - especially if the underlying issue requires a core-code change on either platform - but getting experts from both sides looking at the exact transaction and integration together is usually the fastest path to a definitive answer.