Ask any developer what they'd rather be doing than writing ATF tests, and the answer is almost always: building. Authoring a single, carefully assembled ATF test typically takes 15 to 20 minutes, a modest feature needs five to seven of them, and troubleshooting one failing test can burn 2 to 3 hours. I opened a recent Platform Academy session with those numbers because they explain exactly why ATF Test Agent exists.
On May 28, 2026, I presented "Introducing ATF Test Agent" for Platform Academy, with a live demonstration from my colleague Ravi Mantrala, alongside Sharon Barnes. The full recording is available on YouTube, linked again at the end of this post.
In this post
The Road to Test Agent
This session picked up where our March 31, 2026 Platform Academy session on Agentic ATF left off. That session introduced two capabilities that are now generally available — ATF Code Coverage, which gives line-by-line visibility into what a test suite exercises (green for hit, red for missed), and the ATF Test Troubleshooting Agent, which performs automated root-cause analysis on failing tests. Both of those already existed by the time I sat down for this one. What was still missing was authoring and orchestrated execution inside that same conversation.
ATF Test Agent is the piece that closes that loop. It brings test authoring, execution, and troubleshooting into the same Build Agent conversation used to build an application, and it's generally available from the Australia release. The recording and post from that March session are linked in the Resources section at the end of this one.
Authoring
Why it matters: authoring a single test by hand takes 15 to 20 minutes, and a modest feature needs five to seven of them. That's the single biggest time sink in the whole testing lifecycle, and it's the one most developers actively avoid.
What's shipping: authoring starts from a single natural-language prompt typed into the same chat window used to build the app. The agent shares that build conversation's full application context, so the codebase does not need to be provided separately: it reads the code and the intended business logic directly, then generates tests across every classification path and business rule it identifies. Generated tests are saved to the standard sys_atf_test table, the same table every hand-built ATF test uses, so existing ownership, naming, and review workflows carry over unchanged.
What this means for customers: Ravi demonstrated this using a small custom-scoped app built in Build Agent: a coffee-ordering catalog item with a business rule making an employee's first black coffee of the day free and $3 after that, set prices for a latte and a cappuccino, and an email notification on order. With the app open in the Build Agent conversation, he typed one prompt asking the agent to write ATF tests for all of the feature permutations in the app. The agent generated ten ATF tests without further edits or re-prompting: pricing checks for each coffee type (including the $0 first-black-coffee case and the $3 charge after), the free-first-order rule specifically, and a check that the order confirmation email sent. All ten tests appeared immediately in the standard ATF module, saved to sys_atf_test exactly like a manually built test, representing work that would otherwise take roughly 15 to 20 minutes per test.
Build Agent chat showing the single authoring prompt and the ten ATF tests it generated, saved to the standard ATF module.
This capability is not limited to custom-scoped apps or net-new builds. One attendee asked whether it works for existing forms and catalog items built in Studio, not just apps generated from scratch. It does, with one gating factor: it works in global and store scopes too, and on applications that already existed on the instance, as long as the underlying metadata is understood by Fluent, ServiceNow's proprietary TypeScript-based language for defining application metadata on the backend. Forms, catalog items, lists, and email elements are all currently within Fluent's scope. A practical test offered during the session: if an item opens in the ServiceNow IDE (from the drop-down at the top left of the screen), Fluent understands it, and the Test Agent can work with it.
Scope requirement: Authoring works in custom, global, and store scopes, and on apps that already existed before Test Agent did, not only net-new builds, provided the metadata is understood by Fluent.
Execution
Why it matters: generating tests is only half the job if running them still means leaving the conversation for a separate module or standing up your own test infrastructure.
What's shipping: execution is triggered by a follow-up prompt in the same conversation used for authoring, without a separate runner setup or a different module to navigate to. The agent submits the tests to Cloud Runner, ServiceNow's own hosted execution infrastructure, so no headless browser or local test runner needs to be configured separately. Cloud Runner executes the tests against the instance's live configured data and returns a structured pass-and-fail table, with failures flagged, in the same conversation window.
What this means for customers: in the coffee-ordering demo, Ravi's follow-up prompt asked the agent to execute the tests. The agent used context from the conversation to identify "the tests" as the ten it had just generated, submitted them to Cloud Runner, ran them in sequence, and returned results in the same chat window rather than a separate test-run screen.
Cloud Runner's structured pass/fail results table returned inline in the Build Agent conversation, with the failing test flagged.
Two attendees asked about the mechanics and cost of this execution layer. The Test Generator on Cloud Runner app is a free store app that includes two Cloud Runner lanes at no charge. A lane is a parallel execution slot, a worker that can run one test at a time; with two free lanes, two tests run simultaneously while the rest queue behind them. Purchasing additional lanes increases how many tests run at once, which reduces wall-clock time for a large suite rather than changing what any individual test does. A separate question asked about alternatives to Cloud Runner for protecting test data. For cloud customers, Cloud Runner is ServiceNow's own infrastructure rather than a third party, so test data is held to the same trust model as the rest of the instance. I shared knowledge base article KB1188991, which covers Cloud Runner data handling specifically, in the session chat for anyone who wants it in writing.
Setup requirement: Execution depends on the Test Generator on Cloud Runner store app being installed on your instance; it is free and includes two Cloud Runner lanes, with additional lanes available for purchase for parallel runs.
Troubleshooting and Maintenance
Why it matters: troubleshooting a single failing test can burn 2 to 3 hours, the second-biggest time cost after authoring, and it's the step where the "agentic" part of ATF Test Agent does the most work.
What's shipping: when a test fails, the agent's troubleshooting logic engages automatically. It reads the execution logs, identifies the likely root cause, determines whether the fix belongs in the test or in the application code, applies it at the layer that is actually broken, and reruns until the suite is clean. Human review remains the gate before production reliance: the agent performs the analysis and applies the fix, while the team retains judgment over what counts as a passing test.
What this means for customers: in the demo, one of the ten generated tests failed, a pricing check. The agent read the failure detail and identified a likely cause on its own. A tool interruption during the live session prevented the audience from seeing the automated fix and rerun complete in real time, but Ravi confirmed that triage, fix, and rerun is how the loop is designed to behave once the fix is applied.
The agent's root-cause explanation for the failed coffee-pricing test, identifying which layer it flagged as the fix point.
The maintenance side of this loop has a defined limit today. One attendee asked directly whether the agent automatically updates tests as the application changes, or whether a user has to prompt it. Currently it is the latter: if a new pricing rule is added to the coffee app at a later date, the affected tests need to be updated through an explicit prompt rather than the agent detecting the change on its own. I cover where that's headed in the What's Next section below.
Current limit: The agent does not watch for application changes on its own; after you modify an app, you must explicitly prompt it to update the affected tests.
"The ATF Test Agent doesn't replace anything you already have. It unifies the pieces into a single flow inside the conversation you're already using."
Key Takeaways
From Build to Tested: One Continuous Flow
|
💬
Test Agent
Author & Execute |
→
|
📊
Code Coverage
See What's Tested |
→
|
🔍
Troubleshooting Agent
Fix What Fails |
|
All three run inside the same Build Agent conversation
↻ Loops back to building, not a separate workflow
|
A poll asking which of the three areas would deliver the most value first (n=39) found the room decisive: 67% chose authoring, ahead of troubleshooting at 23% and execution at 10%. The points below summarize what the session demonstrated across the three areas of functionality:
- Authoring addresses the highest-cost manual step. A single natural-language prompt produced ten tests covering pricing logic, a conditional business rule, and a notification check, work that would otherwise run 15 to 20 minutes per test if built by hand. The agent identifies permutations directly from the classification paths and business rules it reads in the app, rather than relying on a developer's manual enumeration of cases.
- Coverage extends to existing apps, within defined boundaries. Because authoring works on existing forms, catalog items, and lists, not only net-new apps, teams do not need to wait for a rebuild to get test coverage on what is already in production use, provided the metadata is Fluent-readable. This does not extend to testing an agentic workflow's own decision-making: Agent Evaluations is the appropriate tool for validating an agentic workflow such as a Now Assist skill or AI agent while it is being developed, while ATF is suited to checking the expected, consistent outcomes that workflow produces on records once it is live, a distinction raised directly in the session's Q&A.
- Automated fixes still route through existing production governance. My guidance on this has not changed with this release: autonomous troubleshooting inside the conversation does not change the rules for getting ATF into production. I still recommend moving tests to production for storage only, not execution, since running tests live risks side effects such as real notifications firing. Deactivate tests before promoting an update set, and reactivate only after cloning down to sub-production.
What's Next
A look at what's still unresolved or in motion on this capability. Directional, subject to change.
Automatic, incremental test maintenance. Today, the agent does not watch for application changes on its own; after you modify an app, you have to explicitly prompt it to update the affected tests. Ravi noted during the session that more incremental maintenance, where the agent recognizes what changed in the app and updates only the affected tests instead of regenerating everything, is on the near-term roadmap but not yet available.
Getting Started
If you are weighing whether to pilot this, or need to justify the effort to leadership the way one attendee asked about, the sequence below is the one we walked through on the call.
- Confirm your licensing. A Creator Pro Plus subscription provisions Now Assist for Creator, a single bundled entitlement that also covers Build Agent itself, the ATF Troubleshooting Agent, application generation, flow recommendations, app summarization, and UI generation. If your org already has Creator Pro Plus for other AI-assisted development work, the Test Agent comes with it at no separate cost.
- Confirm your environment. You need App Engine Studio with Build Agent activated on the target instance. ATF Test Agent is generally available from the Australia release (the Q2 2026 family release); the unified single-panel experience demoed in this session shipped in May 2026 and requires Australia Patch 2 or later, or Zurich Patch 9 for orgs backporting without moving to Australia yet.
- Install the Test Generator on Cloud Runner store app. This is what enables execution; it is free and includes two Cloud Runner lanes to start.
- Pick a pilot. Choose one bounded-scope application, ideally something already built in Studio, like the coffee-ordering catalog item in the demo, for a first end-to-end authoring, execution, and troubleshooting cycle.
- Schedule a human review pass. Review the first round of agent-generated tests before relying on them, the same judgment call teams already make on hand-built tests.
- Roll forward deliberately. Expand to broader teams once the pilot review is clean. Socializing results with QA and dev leads before scaling further is a reasonable default — a follow-up poll on next steps found 28% of attendees planned to do exactly that.
"With natural language, you're only limited by your imagination, so let your imagination drive you to where the future is."
Resources
Watch the Full Session
The complete Platform Academy session, including the full live demo on Build Agent, is available on YouTube.
Questions or feedback? Drop them in the comments below. I read every one.
