Ritesh Shah AI
ServiceNow Employee

 

AI Agents: From Build to Production

Building an AI Agent is only half the work — getting it ready for production is where the real discipline begins.

Overview

In this session, Ritesh Ramesh and Dustin Snell walk through practical guidance for improving AI Agent prompts, selecting the right tools, understanding retrieval-augmented generation, and preparing agents for enterprise deployment with guardrails, security controls, and evaluations.

Learn how ServiceNow AI Agents can operate across conversational channels, connect to enterprise systems, use internal and external data sources, and be tested before going live.

Video Timestamps

00:04
AI Agents across channels and enterprise systems
00:53
Prompts and tools: the two core building blocks
01:16
Retrieval-Augmented Generation (RAG) explained
01:46
Context stuffing and context window tradeoffs
02:16
Decomposing prompts and simplifying instructions
02:46
Writing prompts for different models
03:14
Defining clear agent roles
04:21
Prompting best practices recap
04:51
Tools as the second major control
05:41
Knowledge Graph as an AI Agent tool
06:04
MCP tools and external system connections
06:23
Search Retrieval and AI Search
07:27
Reusing Integration Hub connectors as tools
07:49
Preparing agents for production
08:48
Prompt injection and jailbreaking protection
09:10
Data privacy and sensitive information masking
09:51
Agent evals and production testing
10:43
Measuring accuracy and tool usage
11:33
Running evals with ground truth or production logs
11:58
Human-in-the-loop review and regulated industries
12:42
Homework: run an agent eval
13:26
Why evals, KPIs, and measurement matter
13:43
Filtering sensitive topics

What You'll Learn

How AI Agents can operate across channels like Messenger, Slack, Teams, SMS, and more

Why prompts and tools are the two core controls when building AI Agents

How Retrieval-Augmented Generation improves accuracy with business context

When to use context stuffing — and when it can become too much

How to simplify prompts so agents follow instructions more reliably

Why agents need permission to say "I don't know"

How roles, examples, and prompt structure affect output quality

What tool types are available inside ServiceNow AI Agent Studio

How to prepare agents for production with security, privacy, and evals

Tools Covered

🔎

Search Retrieval & AI Search

Find and retrieve relevant information efficiently

📚

Knowledge Articles

Leverage internal and external content sources

🕸

Knowledge Graph

Connect entities and relationships for context

⚙️

Flows & Scripts

Build automations with flows, subflows, and scripts

🔗

Integration Hub

Reuse existing connectors as agent tools

🌐

MCP Tools

Connect to external systems and services

Production Readiness, Security & Evaluations

Deploy AI Agents confidently with these production-grade capabilities:

Now Assist Guardian — Guardrails and safeguards
Offensiveness Detection — Block harmful content
Security Controls — Prompt injection and jailbreaking protection
Data Privacy — Sensitive information masking
Agent Evaluations — Test agents before production
Accuracy Measurement — Validate agent performance
Tool Usage Validation — Ensure correct tool selection
Human-in-the-Loop — Review and oversight for regulated industries

Agent evals help teams prove whether an AI Agent is actually doing its job before turning it on for employee-facing or customer-facing use cases.

💡 Key Takeaway

Good AI Agents are built with both creativity and control.

  • Prompts help define how the agent thinks
  • Tools determine what the agent can do
  • Guardrails define what the agent must not do
  • Evals prove whether the agent is ready

The future of AI Agent deployment will not be just about building agents — it will be about measuring, governing, and improving them continuously.

About This Series

This series explores AI Agents in ServiceNow — from use case selection and architecture to hands-on implementation, prompting, tool selection, governance, evaluations, and production readiness.

Built for ServiceNow architects, developers, administrators, platform owners, AI practitioners, and business leaders looking to build enterprise AI solutions that are useful, measurable, and safe to deploy.

#ServiceNow #AIAgents #AgenticAI #NowAssist #AIAgentStudio #EnterpriseAI #PromptEngineering #AITrust #WorkflowAutomation #ServiceNowDeveloper

ServiceNow AI Agents: From Build to Production

Building enterprise AI solutions that are useful, measurable, and safe.

Version history
Last update:
58m ago
Updated by:
Contributors