Testing what you built
Summarize
Summary of Testing what you built
Test Agent enhances the development process in ServiceNow by automating test creation, execution, and failure analysis for code generated by Build Agent. It enables a seamless build-to-test workflow within a single development session without manual intervention in test authoring or troubleshooting. By generating functional Automated Test Framework (ATF) tests, executing them, and performing root cause analysis (RCA) on failures, Test Agent helps ensure that every build is verified and safe before release.
Show less
Key Features
- Autonomous Test Authoring: Automatically generates functional ATF tests based on the code changes made by Build Agent using the same prompt and context.
- Automated Test Execution and Failure Triage: Runs generated tests and conducts RCA on any failures, providing either automatic fixes or actionable guidance directly within ServiceNow Studio or the ServiceNow IDE chat panel.
- Auto-Healing Loop: Build Agent ingests RCAs from Test Agent, applies fixes autonomously, and re-executes tests until all pass, eliminating manual test maintenance during a development session.
- Integration and Storage: Generated ATF tests are stored in the sysatftests table within the app scope and can be used for scheduled regression testing.
Key Outcomes
- Streamlined Development and Testing: Enables building and testing in one continuous session without switching contexts between coding and test creation.
- Faster Failure Resolution: Automated RCA and proposed fixes minimize time spent investigating test failures.
- Improved Release Confidence: Automated test execution provides measurable quality gates and verifiable code health before promoting to production.
Workflow for ServiceNow Customers
ServiceNow developers use Build Agent in ServiceNow Studio or the IDE to create or edit applications. Test Agent then automatically generates relevant functional ATF tests from the same prompt and code context. Build Agent runs these tests, with Test Agent handling failure triage and RCA. Safe fixes are applied automatically or suggested for manual resolution, followed by re-execution of tests until successful completion.
Scope and Requirements
- Authoring Environments: ServiceNow Studio and ServiceNow IDE.
- Application Scopes: Global, custom, and store applications.
- Test Types: Functional ATF tests.
- Execution Target: Cloud Runner lanes (requires ATF Test Generator and Cloud Runner apps installed and a cloud user configured).
Practical Use
ServiceNow customers can prompt Build Agent with commands such as “Generate ATF tests for all the feature permutations on the app we built” and “Execute all ATF tests” to automate testing workflows. Test Agent’s integrated RCA and auto-heal functionality significantly reduces manual effort in test maintenance and failure investigation, streamlining development cycles and increasing confidence in code releases.
Test Agent generates test coverage for code created by Build Agent, executes tests, and performs root cause analysis on failures. Prompt Test Agent to complete build-to-test workflows in a single development session without manual test authoring or failure investigation.
Test Agent extends Build Agent by making every build safe before release. After Build Agent produces code changes in a development instance, Test Agent consumes the same prompt and code context to author functional Automated Test Framework (ATF) tests, execute those tests, and triage any failures automatically.
For complete documentation on using Test Agent, see Test Agent.
When a test fails, Test Agent performs a root cause analysis (RCA) and either auto-applies safe fixes or surfaces actionable guidance in the chat panel so you can resolve the issue without leaving ServiceNow Studio or the ServiceNow IDE.
Customer outcomes
- Build and test in one session: You no longer need to context-switch between authoring code and writing tests. Both happen sequentially within the same Build Agent session.
- Faster failure triage: Automated RCA and proposed fixes reduce the time you spend looking through logs after a test run.
- More release confidence: Measurable quality gates enforced by automated test execution give you verifiable evidence of code health before promotion to production instances.
- Generated ATF tests are stored in the sys_atf_tests table under the app scope for which they were created. You can schedule regression test runs using the generated tests.
Example prompts
An example prompt to get Build Agent to create and run ATF tests would be Generate ATF tests for all the feature permutations on the app we built.
Then, you can tell Build Agent to Execute all ATF tests.
Test Agent workflow
The end-to-end workflow is:- Create or edit an app in a development instance using Build Agent in ServiceNow Studio or the ServiceNow IDE, driven by your prompt.
- Test Agent consumes the prompt and the resulting code changes to generate contextually relevant functional ATF tests.
- Prompt Build Agent to run the tests.
- Failures are automatically triaged. Test Agent produces an RCA and either applies safe fixes autonomously or proposes them to you through the Build Agent chat panel.
- Build Agent ingests the RCA from Test Agent and re-executes tests until a passing status is achieved, completing the auto-heal loop.
Key developer experiences
- Autonomous test authoring
- When you use Build Agent to implement a new story, you can prompt it to create functional tests automatically.
- Assisted troubleshooting
- Test Agent automatically troubleshoots failed tests, generates RCAs, and proposes targeted fixes, eliminating manual log investigation.
- Auto-healing
- Build Agent consumes the RCA produced by Test Agent and applies fixes to code or tests, then re-executes the test suite until all tests reach a passing status. This removes the need for developers to manually patch and maintain tests during a session.
Scope and availability
Test Agent is available in the following environments and scopes:
| Dimension | Supported values |
|---|---|
| Authoring environment | ServiceNow Studio, the ServiceNow IDE |
| Application scope | Global, custom, store |
| Test types | ATF functional tests |
| Execution target | Cloud Runner lanes |