AI Gateway FAQ

Ashley Snyder · 2 hours ago

📋 Quick Overview

AI Gateway is the centralized governance layer within AI Control Tower for all MCP server connections across agent platforms. This FAQ covers everything from getting started and key concepts to security, compliance, and using AI Gateway day-to-day.

❓ Getting Started

What is AI Gateway?

As organizations scale AI agent deployments across multiple platforms, the connections between those agents and external tools multiply quickly — and so does the risk. Without a central control point, it becomes difficult to know which agents are accessing what, whether those connections are properly authenticated, and what's happening across all of them in real time.

AI Gateway addresses this through four capabilities that work together:

Discover — A centralized registry of every MCP server in your environment, whether auto-discovered from AI Agent Studio or imported from the community catalog with one click. Every server follows a defined lifecycle from intake to retirement, with a complete audit trail.
Govern — An approval workflow for every MCP server, managed through AI Control Tower's standard Asset Approval Playbooks. Once approved, servers get a secure Gateway URL — no direct agent connections, ever. In Q1 2026, those approvals are enforced directly in AI Agent Studio so builders only see what's been approved.
Control — OAuth 2.1 authentication enforced on every connection — not recommended, enforced. AI Stewards can pause any server instantly, individually or globally. Automated PII detection blocks sensitive data at the gateway before it reaches logs or downstream systems. In Q1 2026, 10+ platforms including Microsoft Copilot Studio, AWS Bedrock, and Google Vertex AI auto-configure through CIMD — no manual credential setup required.
Observe — Usage, latency, and success rates at the server and tool level, generated automatically on every call. In Q1 2026, client-level connection analytics let AI Stewards track success and failure rates by MCP client and filter by time period — making it faster to identify which platform is experiencing issues and when.

Why do I need AI Gateway?

As organizations scale AI agent deployments across multiple platforms, the connections between those agents and external tools multiply quickly — and so does the risk. Without a central control point, it becomes difficult to know which agents are accessing what, whether those connections are properly authenticated, and what happened when something goes wrong.

AI Gateway addresses three specific challenges:

Visibility — A single registry showing which agents access which tools across your entire organization, so you're never in the dark about what's connected to what.
Security — Centralized policy enforcement and authentication management, so your security team doesn't have to chase configurations across a dozen platforms.
Compliance — Complete audit trails for every agent interaction, so you can respond quickly to incidents and meet regulatory requirements without manual log-digging.

Is AI Gateway available now, or is this a future product?

AI Gateway is available today. The initial release shipped in December 2025, and the Q1 2026 release (March 2026) is also generally available, bringing significant enhancements to governance, security, and observability.

If you're on Zurich Patch 4 or later with an AI Control Tower Core or Pro Plus SKU, you can get started right now. The product documentation at servicenow.com/docs covers installation prerequisites and setup end to end, and your ServiceNow account team can help you map out an implementation path specific to your environment.

How much does AI Gateway cost?

For specific pricing details and how usage-based costs apply to your subscription, reach out to your ServiceNow account team.

How do I get access to AI Gateway?

AI Gateway is included with AI Control Tower — there's no separate purchase or activation required. If your organization has AI Control Tower entitlements, AI Gateway is automatically available on your instance.

To use AI Gateway, you need:

Zurich Patch 4 or later
AI Control Tower Core 6.1.0 (Q1 features)

Does AI Gateway work if I'm not using ServiceNow AI Agents?

Yes — and this is an important point. AI Gateway is platform-agnostic by design. It governs connections from any agent platform, not just agents built in ServiceNow AI Agent Studio.

If your organization is running agents on Microsoft Copilot Studio, AWS Bedrock, Google Vertex AI, or a custom-built agent platform, those agents can connect to MCP Servers through AI Gateway using the standard Gateway URL and OAuth 2.1 authentication. From a governance perspective, the platform the agent was built on doesn't matter — what matters is that the connection flows through AI Gateway, where it can be approved, authenticated, monitored, and controlled.

In Q1 2026, connecting external platforms became significantly faster. Platforms that support CIMD can auto-configure through a single registration in AI Control Tower, covering all approved MCP Servers on that host without repeated per-server setup. For platforms that don't yet support CIMD, guided OAuth 2.1 setup is available for manual configuration.

This is especially relevant for organizations with multi-vendor AI strategies, where different teams may be building on different platforms. AI Gateway gives your security and governance teams a single place to oversee all of it.

📖 Understanding Key Concepts

📖 Key Terms

MCP (Model Context Protocol): An open-source industry standard that defines how AI agents connect to external tools and systems — like USB-C for AI.
MCP Server: A program that exposes capabilities (tools, APIs, data sources) to AI agents in a standardized way.
AI Steward: A governance, security, or compliance team member who reviews and approves MCP Server requests in AI Control Tower.
CIMD: Client Identity Metadata Document — allows agent platforms to auto-register with AI Gateway using their HTTPS certificate, eliminating per-server OAuth setup.

What is Model Context Protocol (MCP)?

MCP is an open-source industry standard that defines how AI agents connect to external tools and systems. Think of it like USB-C for AI — instead of every agent platform building its own custom integration for every tool it needs to access, MCP provides a universal connection standard that works across platforms.

In practical terms, MCP standardizes two things: how an agent discovers what tools are available, and how it invokes those tools. A GitHub MCP server, for example, exposes a set of capabilities — search repositories, create issues, read files — and any MCP-compatible agent can discover and call those capabilities without needing a custom integration built specifically for that agent platform.

For enterprises, this matters because it means your AI agents can access a growing ecosystem of external tools and systems without a proliferation of one-off integrations to manage, secure, or audit.

What is an MCP server?

An MCP server is a program that exposes a set of capabilities — tools, APIs, data sources — to AI agents in a standardized way. It tells any MCP-compatible agent what it can do and how to call it.

Examples of what an MCP server might expose: the ability to search a knowledge base, create a ticket, query a database, or retrieve a file from Google Drive. The MCP server handles the connection to the underlying system; the agent just needs to know the server exists and what tools it offers.

In the context of AI Gateway, every external tool your agents connect to is registered as an MCP Server. AI Gateway governs which MCP Servers are approved for use, how clients authenticate to them, and what happens at runtime when an agent makes a call.

What is the relationship between AI Gateway and AI Control Tower?

AI Control Tower is ServiceNow's complete AI governance platform — it's where you manage AI models, agents, policies, risk assessments, and compliance across your enterprise. AI Gateway is a feature set within AI Control Tower that specifically handles cross-platform agent connections using MCP protocols.

A useful way to think about it: AI Control Tower is the broader governance platform for your entire AI program. AI Gateway is the component within it that focuses on the connections your agents make to the outside world.

You don't purchase or install AI Gateway separately. It's included with AI Control Tower Core and Pro Plus, and the plugin (sn_awh_config) installs automatically when you set up AI Control Tower.

How is AI Gateway different from an API Gateway?

API Gateways manage traditional application-to-application calls — they're designed for REST APIs, microservices, and similar request/response patterns between software systems. They're excellent at what they do, but they weren't built with AI agents in mind.

AI Gateway is purpose-built for agentic workloads. That means it understands MCP as a protocol, manages the specific lifecycle of agent-to-tool connections (intake, approval, authentication, runtime enforcement), and provides the kind of audit trail and policy controls that enterprise AI governance requires. It also handles things an API Gateway doesn't need to think about — like detecting sensitive data in agent requests before they reach an external tool, or enforcing that only approved servers are available to agent builders at development time.

💡 If your organization already uses an API Gateway, AI Gateway sits alongside it, not in place of it. They solve different problems.

How is AI Gateway different from an LLM Gateway?

An LLM Gateway manages access to AI models themselves — it routes requests to foundation models like GPT-4, Gemini, or others, and typically handles things like model selection, rate limiting, and cost tracking at the model layer.

AI Gateway operates at a different layer entirely. It doesn't govern which model your agent uses — it governs what external tools and systems that agent can access once it's running. Think of it this way: your LLM Gateway determines the brain your agent thinks with; AI Gateway determines what that agent is allowed to reach out and touch.

ServiceNow itself uses an LLM Gateway when you connect to ServiceNow-integrated model providers through the platform — that's what's handling your model routing and access at the AI layer. AI Gateway is what governs the tool connections your agents make from there. Both are active in a fully deployed ServiceNow AI environment, operating at different layers of the stack.

⚙️ What AI Gateway Does

What are the main capabilities of AI Gateway?

AI Gateway is built around four capabilities that work together as a closed loop: Discover, Govern, Control, and Observe. Each one addresses a specific question your security and architecture teams will ask the moment you start deploying AI agents at scale.

1. Discover — How do you know what's connecting?

AI Gateway gives you a centralized registry of every MCP Server in your environment, whether discovered automatically or registered manually. MCP Servers created in AI Agent Studio are auto-discovered and synced to AI Control Tower. In Q1 2026, Product Owners can also browse 500+ vetted servers from the community catalog and import them with one click — details pre-populated, no manual entry required.

Every server follows a formal lifecycle: In Review → Assess → Approved → Deployed → Deprecated. Your AI Stewards always have a complete, current picture of what's in the environment, what stage each server is in, and what's pending review.

2. Govern — How do you make sure nothing runs without approval?

Nothing moves to Active without an AI Steward reviewing and approving it. That review happens through AI Control Tower's standard Asset Approval Playbooks — the same governance workflows used for other AI assets like models and datasets — so your team isn't learning a new process.

In Q1 2026, those approvals are enforced directly in AI Agent Studio. Builders only ever see servers that have been approved — unapproved and rejected servers are not visible. Every approved server is part of a CMDB-integrated inventory with a complete audit trail from intake to retirement.

3. Control — How do you enforce governance at runtime?

Approval without enforcement is just documentation. AI Gateway makes governance meaningful at runtime in three ways.

First, every connection is authenticated via OAuth 2.1. Agents never connect directly to an MCP Server — all traffic flows through AI Gateway, which enforces authentication on every call. In Q1 2026, 10+ agent platforms can auto-configure through CIMD, reducing what was previously a multi-step OAuth setup to a guided experience.
Second, if a security issue arises, AI Stewards can instantly pause any MCP Server — individually or globally — directly from AI Control Tower. All traffic stops immediately, with no deployments or code changes required.
Third, when traffic contains sensitive data, AI Gateway's integration with ServiceNow PII Vault Service blocks it at the gateway before it reaches the MCP Server, logs, or downstream systems. No code changes or developer training required. NEW Q1 2026

4. Observe — How do you know if something goes wrong?

Every call through AI Gateway automatically generates metrics — usage, latency, success rate, and which tool was invoked. No additional instrumentation or custom pipelines needed.

You can view that data at the server level for a high-level health picture, or drill into individual tools to see P50, P90, and P95 latency and success rates. That tool-level granularity matters: when something is slow or failing, you can pinpoint exactly which tool within a server is the problem without digging through logs across multiple systems.

In Q1 2026, client-level connection health tracking lets AI Stewards track successful and failed connections by MCP client and filter by time period — making it faster to identify which platform is experiencing issues and when the problem started.

Because every transaction is logged, your compliance team has a complete audit trail — who connected to what, when, and with what outcome — without any reconstruction effort.

🤖 Using AI Gateway

Who are the main roles in AI Gateway?

AI Gateway is designed around two personas who each interact with it differently:

Product Owners are the builders — the people creating AI agents and connecting them to external tools. Their primary interaction with AI Gateway is through AI Agent Studio, where they submit requests to add MCP Servers, configure which tools their agents use, and view usage metrics for their connections. In Q1 2026, Product Owners working in AI Agent Studio will only see approved MCP Servers in their dropdown — unapproved servers are hidden, so governance is enforced at the point of development, not after the fact.
AI Stewards are the governors — typically members of your AI governance, security, or compliance teams. They review and approve MCP Server requests, configure authentication, enable or disable servers, activate sensitive data protection, and monitor connection health across the enterprise. AI Stewards work primarily in AI Control Tower.

Most users fall into one of these two roles. Your ServiceNow administrator can assign the appropriate role — contact them if you're unsure which one applies to you.

How do I submit a request to add an external tool?

If you're a Product Owner building an agent in AI Agent Studio, you can submit an MCP Server request directly from the Studio. Navigate to Settings > Manage MCP Servers, then click New. From there you can either enter the server details manually or, in the Q1 2026 release, browse and import from the MCP community catalog with one click — details are pre-populated so there's no manual configuration.

Your request enters the approval lifecycle in AI Control Tower, where an AI Steward reviews it. You'll be able to track the approval status on the MCP Server record.

If you're registering a server that wasn't created in AI Agent Studio — for example, an external third-party server your agents need to access — that intake can also be done directly through AI Control Tower in the Q1 2026 release, without needing to go through AI Agent Studio first.

What authentication methods are supported?

AI Gateway supports OAuth 2.1 as its primary authentication method. Every MCP Server connection using OAuth 2.1 routes through the AI Gateway proxy, where token validation and rotation are enforced centrally — agents never connect directly to the server.

MCP Servers configured with API key authentication can also be registered and synced into AI Control Tower, where they go through the standard governance lifecycle — intake, review, and approval. However, API key servers do not currently route through the AI Gateway proxy in the same way OAuth 2.1 servers do.

⚠️ For enterprise connections where full runtime enforcement is required, OAuth 2.1 is the recommended approach.

How does automated client registration work for external platforms? NEW Q1 2026

Before Q1 2026, connecting an external agent platform to AI Gateway required a separate OAuth client registration for every MCP server that platform needed to access. A team using Copilot Studio with ten approved servers had to repeat the full setup process ten times.

Q1 2026 introduces automated client registration through CIMD (Client Identity Metadata Document), a capability introduced in the MCP specification that allows a client to prove its identity dynamically using its HTTPS certificate rather than requiring pre-registration on each server.

In practice, an AI Steward registers the external platform once in AI Control Tower by entering the platform's CIMD client URL under Configurations > AI Gateway. From that point, a single registration covers every MCP server on that host — when a new server is approved, it automatically becomes accessible to that client without any additional setup.

For platforms that don't yet support CIMD, guided OAuth 2.1 setup remains available for manual client credential configuration.

How do I troubleshoot connectivity issues across platforms?

AI Gateway logs every connection attempt with the client identity and timestamp. In Q1 2026, AI Stewards can use this data to track down connectivity issues by counting successful and failed connections, filtering by time period, and grouping results by MCP client.

This makes it faster to identify which agent platform is experiencing issues and when the problem started — without having to cross-reference logs from multiple systems.

For deeper investigation once you've identified the affected client or time window, your next step would be reviewing that server's configuration and authentication setup in AI Control Tower. If the issue warrants immediate action, any MCP server can be paused instantly while you investigate.

How does an AI Steward approve or reject an MCP Server?

When a Product Owner submits an MCP Server request, it appears in AI Control Tower with a status of In Review. To review it, navigate to AI Control Tower > AI assets > MCP Servers and filter to servers in the In Review state. The AI Steward opens the server record and works through the Asset Approval Playbook — reviewing what tools the server exposes, what data it accesses, whether it comes from a trusted source, and whether it meets your organization's security requirements. At the end of the playbook, the Steward approves or rejects the request.

Approved servers move to the Deployed lifecycle phase and become available to agents. Rejected servers cannot be added to agent configurations. The entire review history is logged, so there's always a record of who approved what and when.

Can I pause a server if something goes wrong?

Yes. AI Stewards can pause any MCP Server instantly — individually or globally — directly from the AI Gateway setup tab on the MCP Server record in AI Control Tower. When a server is paused, all agent traffic to that server stops immediately. Agents attempting to call a paused server will receive an error response. No code changes or deployments are required, and the server can be resumed just as quickly once the issue is resolved. Configurations are preserved when a server is paused, so no setup needs to be repeated to resume.

What do agents experience when a server is paused?

When an AI Steward pauses an MCP server — individually or globally — all traffic to that server stops immediately. Agents attempting to call a paused server will receive an error response indicating the server is currently unavailable. No agent-side code changes are required.

Configurations are preserved when a server is paused. Once the AI Steward resumes the server, it becomes available again immediately — agents connecting after resumption will succeed without any reconfiguration on the agent side.

This is intentional by design: the pause mechanism is built for speed in security or compliance scenarios, and the resume path is equally fast so the impact on agent users is as short as possible.

Does AI Gateway support locally hosted MCP Servers?

No — AI Gateway only supports MCP Servers hosted on remote servers. Locally hosted servers running on a developer's machine are not supported. This is by design, as remote hosting is required to ensure the secure, reliable connections that enterprise governance depends on.

🔒 Security and Compliance

How does AI Gateway enforce security at runtime?

Security in AI Gateway operates across three controls that apply on every MCP call.

1. Server State

Before anything else, AI Gateway checks whether the MCP Server is currently active. If an AI Steward has paused the server — globally or individually — the request is blocked immediately.

2. Authentication

Every connection is routed through AI Gateway and OAuth 2.1 is enforced — not recommended, enforced. There is no path to an MCP Server that bypasses authentication. AI Gateway manages token validation and rotation centrally, so there's no risk of hardcoded credentials or tokens that never rotate.

3. PII Detection

If an AI Steward has activated PII filtering for that server, every call to that server is scanned by ServiceNow PII Vault Service. If sensitive data is detected, the traffic is blocked entirely — it doesn't pass through, it doesn't get logged downstream, and it doesn't reach its destination.

All three controls are enforced at the infrastructure level. No developer involvement required, no relying on individual teams having implemented their own checks.

How does sensitive data protection work, and what does "blocked" mean exactly?

When an AI Steward activates PII filtering for an MCP Server — one toggle in AI Control Tower — every call to that server passes through AI Gateway and is scanned by ServiceNow PII Vault Service. The scan checks for patterns configured in your system, such as social security numbers, credit card numbers, and other PII types your organization has defined.

If sensitive data is detected, AI Gateway blocks the entire payload. It does not pass through to the agent, it does not appear in downstream logs, and it does not reach any connected system. The blocked event is recorded in the audit trail so your compliance team has a complete record.

This protection is consistent across every agent connecting to that server, regardless of which platform the agent was built on or who built it.

Can I quickly respond if a security issue is detected?

Yes — AI Stewards can pause any MCP Server instantly, either individually or across all servers globally, directly from AI Control Tower. When a server is paused, all agent traffic to it stops immediately. No change management cycle, no deployment window, no code changes required. The server can be resumed just as quickly once the issue is resolved.

This is particularly useful in situations like a vendor disclosing a vulnerability, an unusual spike in call volume, or a server behaving unexpectedly — scenarios where you need to act in seconds, not hours.

Does AI Gateway help with compliance requirements?

AI Gateway contributes to compliance in two direct ways.

First, PII detection and blocking prevents sensitive data from transiting through agent connections to unauthorized systems, supporting data minimization and access control requirements.
Second, every transaction flowing through AI Gateway is logged, giving your compliance team a complete, ready-to-use audit trail — who connected to what, when, and with what outcome — without any reconstruction effort after the fact.