***My team was in a meeting with a CTO last month. He directed his ServiceNow CoEI Head and Director of Engineering work with us to paint a picture around what could be a high performing Enterprise AI Stack. After several discussions & working sessions what came up was fascinating. This article shares some of that information.
Note: This information will provide a baseline for many.This is not a final representation of the approved stack. This article is to help Engineering leaders understand key considerations while they build or upgrade their stacks***
1. Purpose
This document specifies a reference architecture for enterprise AI platforms — the infrastructure required to operate AI development tools and autonomous agents at scale. It identifies seven layers that together provide cost-attributable, governed, and observable AI operations, and describes the role of ServiceNow as the integration spine that ties the layers together.
The intended audience is enterprise architects, platform engineers, and technology leaders responsible for AI tooling rollouts. Component references are accurate as of April 2026; layer abstractions are intended to outlive specific product selections.
2. Key Considerations
Enterprises that have deployed AI development tools — IDE assistants, agentic CLI tools, and code security scanners — commonly encounter operational issues that are architectural in nature rather than tool-specific. The four below are representative of the gaps this architecture addresses:
- Cost attribution. Token consumption grows but cannot be allocated to specific projects, clients, or engagements.
- Security policy enforcement. AI-generated code may reach production without standardized scanning or review.
- Data residency and confidentiality. Source code or regulated content may be transmitted to model providers through developer tooling, outside the scope of acceptable use policies.
- Agent inventory and governance. Organizations cannot reliably enumerate the AI agents running in their environments, including those deployed by third-party SaaS vendors.
These issues require infrastructure layers built around the tools: a gateway for routing and cost control, an orchestration platform for agents, and a governance layer for runtime policy enforcement. No single vendor provides all of these. ServiceNow provides the workflow, governance, and protocol surface that ties them together (Section 5).
3. Architectural Diagram
The reference stack comprises seven layers, ordered from foundational (L1) to outcome-facing (L7). The dependency graph is not strictly linear: L1 is depended on by L2 through L7 for identity; L3 is depended on by L2 and L4 for LLM access; L6 is depended on by L7 for telemetry, and in turn depends on L3 and L4 for the data and behavior it governs.
Two control-plane flows are essential to operation:
- L3 → L6. OpenTelemetry trace data emitted by the model gateway is ingested by AI Control Tower for each LLM call.
- L6 → L4. Runtime policy decisions from AICT are applied back into AI Agent Fabric, where agent activity occurs.
Both flows terminate at ServiceNow products — the architectural feature that makes ServiceNow the integration spine of the stack (Section 5).
4. Layers
Each layer description identifies its purpose, primary components, and ServiceNow's role or integration point.
4.1 L1 — Identity & Access
Identity governance for human and non-human (agent) identities. Primary identity comes from Okta or Microsoft Entra ID. Veza provides entitlement governance, including for non-human identities. AI Control Tower integrates with Veza through its Secure capability, extending least-privilege enforcement to AI agents and systems managed by the platform. All other layers require verified identity to operate.
4.2 L2 — AI-Assisted Coding
Developer-facing AI tools (Cursor, Claude Code, GitHub Copilot) and code security scanning (Snyk) in CI/CD. For ServiceNow application development, the ServiceNow SDK (npm package @Servicenow/sdk, version 4.6.0, released 15 April 2026) enables git-managed local development with standard tooling — tables, business rules, script includes, flows, and UI actions represented as code files. The 4.6.0 release introduced ServiceNow SDK AI Skills, including the now-sdk-explain skill, which gives external AI coding assistants (such as Claude Code) live access to SDK documentation at inference time, reducing dependency on outdated training data.
Gateway
The single-gateway pattern at L3 applies cleanly to API-direct usage and custom-built agents. Commercial IDE assistants vary:
- Claude Code. Routes via the ANTHROPIC_BASE_URL environment variable.
- Cursor. Supports bring-your-own-key and custom endpoints for some models; the IDE itself talks to Cursor's backend.
- GitHub Copilot. Routes through GitHub. Enterprise tier provides policy controls but not request-level gateway routing.
- Amazon Q Developer. Routes via AWS endpoints; PrivateLink and VPC endpoint integration supported.
- Custom agents and API-direct calls. Fully gateway-routable.
Cost attribution and policy enforcement at L3 — and visibility in AICT at L6 — apply with full fidelity to custom agents and API usage, and with partial fidelity to commercial IDE assistants.
4.3 L3 — Model Gateway
A single point of egress for routable LLM calls. AWS Bedrock or Azure API Management (APIM) serves this role; Lakera Guard screens prompts. The gateway provides:
- Per-request cost attribution by project, client, or engagement.
- Policy-based routing, including region-restricted endpoints for regulated data.
- Prompt injection screening.
- Provider abstraction across vendors, to support model substitution without application changes.
- Telemetry emission via OpenTelemetry to AICT at L6.
ServiceNow does not provide the gateway, but the gateway's telemetry is the primary feed for AICT governance. AICT's documented integrations with Anthropic, OpenAI, AWS, Microsoft, and NVIDIA extend governance reach across the model and infrastructure layer.
4.4 L4 — Orchestration
ServiceNow with AI Agent Fabric is the orchestration layer. AI workflows operate inside existing platform processes — change management, approvals, escalations, audit — rather than as a parallel system. AI Agent Fabric supports the Model Context Protocol (MCP, originated by Anthropic) and the Agent2Agent Protocol (A2A, originated by Google). ServiceNow's A2A client and server support shipped in Zurich Patch 4 (Now Assist AI Agents 6.0.x, December 2025) and targets A2A protocol v0.3. Agents built on Salesforce, Microsoft Copilot, Google Vertex, or custom frameworks coordinate with ServiceNow agents through this protocol surface without bespoke integration code.
4.5 L5 — Enterprise Connectors
Integrations to enterprise systems reachable from L4: ERP (SAP), CRM (Salesforce), data platforms (Snowflake, Databricks), source control (GitHub, GitLab), and collaboration (Microsoft Teams, Slack). ServiceNow IntegrationHub is the primary connector framework, augmented by iPaaS where coverage gaps exist. AICT's Discover capability extends inventory and governance to assets reached through these connectors.
4.6 L6 — Governance & Observability
ServiceNow AI Control Tower (AICT) combines runtime policy enforcement with operational telemetry. Introduced at Knowledge 2025 and expanded at the April 2026 Australia release, AICT operates across five capability dimensions as named in ServiceNow's product materials:
- Discover. Inventory of AI agents, models, and identities across ServiceNow and external systems (AWS, Google Cloud, Azure, SAP, Oracle, Workday).
- Observe. Continuous runtime monitoring with live metrics; agent-behaviour observability from the Traceloop acquisition.
- Govern. Policy enforcement, lifecycle workflows (intake, risk assessment, deployment, retirement), and approval routing.
- Secure. Identity access governance with Veza, scoped permissions, and a runtime kill switch for agents operating beyond their permissions.
- Measure. Cost tracking and ROI dashboards for model spend.
Operational observability comes from Dynatrace, fed by OpenTelemetry traces from the gateway using the GenAI semantic conventions. Dynatrace and AICT exchange signals: cost anomalies in telemetry trigger AICT policy events; AICT-detected behavior violations become Dynatrace trace events correlated with infrastructure conditions.
4.7 L7 — Reporting
Aggregated metrics and KPIs derived from the L6 telemetry feed — usage by team and engagement, cost trends, error rates, and ROI against defined baselines. ServiceNow Platform Analytics provides metrics tied to AICT-managed assets; Power BI handles cross-source executive reporting. Integrity at this layer depends on integrity of the telemetry feed from L6 and L3.
5. ServiceNow as the Architectural Spine
ServiceNow does not own every layer of the architecture, but it operates directly at four layers and integrates with the other three. This places ServiceNow in a structurally distinctive position — it ties the stack together without replacing the components at the layers it does not own.
5.1 ServiceNow's footprint across the stack
Table 1 summarizes where ServiceNow appears directly and where it integrates with components provided by other vendors.
|
Layer |
Role |
ServiceNow component |
|
L1 — Identity & Access |
Integration |
AICT Secure capability via Veza |
|
L2 — AI-Assisted Coding |
Direct |
@Servicenow/sdk + now-sdk-explain skill |
|
L3 — Model Gateway |
Integration |
AICT Observe ingests OTel telemetry |
|
L4 — Orchestration |
Direct |
AI Agent Fabric (MCP, A2A) |
|
L5 — Enterprise Connectors |
Direct |
ServiceNow IntegrationHub |
|
L6 — Governance & Observability |
Direct |
AI Control Tower (AICT) |
|
L7 — Reporting |
Direct |
Platform Analytics |
Table 1. ServiceNow's role per layer.
5.2 The two integration products
Two ServiceNow products perform the structural integration work — AI Agent Fabric at L4 and AI Control Tower at L6.
AI Agent Fabric
By implementing MCP and A2A as open inter-agent protocols, AI Agent Fabric removes the need for bespoke integration code between agents from different vendors. Agents built on Salesforce, Microsoft Copilot, Google Vertex, or custom frameworks discover, invoke, and coordinate with ServiceNow agents through the same protocol surface. The multi-vendor agent ecosystem becomes composable rather than custom-integrated; ServiceNow becomes the orchestration plane where heterogeneous agents converge on enterprise workflows.
AI Control Tower
AICT is the discovery, governance, and observability plane that spans the enterprise — not only ServiceNow-native assets. Through its integrations with AWS, Google Cloud, Azure, SAP, Oracle, and Workday, it maintains a unified inventory of AI agents, models, and identities. The ServiceNow CMDB and Common Services Data Model (CSDM) provide the shared data structure that makes cross-vendor inventory coherent. AICT enforces policy and audit logging against this unified inventory regardless of where the underlying asset runs.
Together they account for the two control-plane flows from Section 3: the L3 → L6 telemetry flow lands in AICT; the L6 → L4 policy enforcement flow originates in AICT and applies to agents in AI Agent Fabric. Both flows operate against open protocols and standards.
