This post is directed to those who have been tasked with creating ATF tests for the first time. It is by no means a complete exposition on all the features of ATF, but should be enough to become functionally literate for your first dive into writing tests.
Lisa Holstein also posted a great intro that provides a structured approach to learning ATF, and where to find ServiceNow documentation in a meaningful way. It is worth checking out: Getting Started with Automated Test Framework (ATF) .
Overview
ServiceNow Automated Test Framework (ATF) can be used for various testing scenarios: Functional story validation, Regression, Browser Compatibility, Client and server-side functionality, Performance testing, and Load testing. ATF does not currently incorporate any AI nor a “smart” architecture. The developer is given a minimalist but sufficient stable of testing tools.
Tests and Suites can be run manually from the Run Test button on the Test form. Suites can also be run from a schedule.
OOTB there are many read-only Quick Start (sample) Test Suites and Tests. You can use these as starting points for your own tests. See All ⪢ Automated Test Framework (ATF) ⪢ Quick Start Suites, and Copy Test or Copy Suite as needed.
Roles:
- atf_test_admin configuration, custom test steps
- atf_test_designer create and run tests
Test Organization
Test Suites can be organized in a parent-child hierarchy to create a “suite of suites”. In any child Test Suite, select a Parent Suite from the reference field.
About Test Transactions
Any time tests are run, whether as a Suite or as an individual Test, ATF will wrap the entire operation in a transaction. The transaction is rolled back in its entirety at the end of the test, so you cannot look for data generated during the test after it has completed. See the Debugging section for more information.
Create Tests
Start with a test and add test steps to it. Tests can later be arranged in suites.
Test Suites | All ⪢ Automated Test Framework (ATF) ⪢ Suites | sys_atf_test_suite |
Tests | Related List in the Suite form ⪢ New All ⪢ Automated Test Framework (ATF) ⪢ Tests | sys_atf_test |
Test Steps | Related List in the Test form ⪢ New | sys_atf_step |
The Test Suite
Tests can be added to one or more Suites. From here you can also run the suite, debug it, copy it as a starting point for another suite, or delete the suite.
The Test
Tests contain test-specific test steps. From here you can also run the test, debug it, copy it as a starting point for another test, or delete the test.
Test Templates are a collection of reusable Test Steps that can be incorporated as steps in Tests as needed.
Test Steps
Configuration
ServiceNow has categorized the steps with a description of what each test step does.
There are two types of test steps: UI and Server-side. UI steps have the extra need for a browser in which to execute the step. More on this in the Test Execution section.
The configuration for each kind of Test Step will be different depending on its needs.
Study Existing Tests
Find a test that does something similar to what you want to do, and either Copy Test, or study the test to see how, for example, it prepares back end (server-side) system or data objects for use by client (UI) test steps
Quick Start Suites | All ⪢ Automated Test Framework (ATF) ⪢ Quick Start Suites |
Quick Start Tests | Related List in the Suite form All ⪢ Automated Test Framework (ATF) ⪢ Tests filter |
Referencing Output from a Previous Step
When configuring a step, you can often use the output of some previous step:
UI vs Server
In the description for a test step, it is not always explicitly obvious if it is UI or server:
There will be hints in the description, referring to UI components – this indicates it will require a browser at execution time (see Test Execution below).
In most cases, the system or data objects for a UI test step will need to be prepared prior to using the UI step.
Test Step Do’s and Don’ts
- DO NOT use data that already exists on the instance. This creates testing dependencies on otherwise “not-testing data”, which can cause a host of issues, not the least of which is keeping that data “pure” across instances.
This means:- Do not use the Impersonate User test step.
DO use the Create User test step and check the Impersonate User checkbox. - Do not rely on pre-existing incidents or other records.
DO create a reusable Test Template, or even just create the necessary Test Steps on demand.
- Do not use the Impersonate User test step.
- DO use server-side Test Steps whenever possible. Using UI Test Steps requires a user to attend the execution (unless you have spent the time and money to set up for headless browser execution). UI Test Steps also have the disadvantage of being orders of magnitude slower than server-side steps.
- Identify tables from your update set records and use the Record Query test step to confirm that the record exists and is correct (based on the Conditions populated in the test step).
- Use UI test steps only when user interface elements must be verified vs assuming it works as long as the back-end records are there.
Test Execution
Run Test / Suite
Individual Suites and Tests can be executed separately as their own sub-unit for testing purposes, or the entire Suite hierarchy can be executed from the parent Suite.
They can be executed in various modes, either:
- Manual / attended testing: Firefox, Chrome, Safari, Edge, or a Cloud Runner Browser. When you execute the test, you are prompted to select a browser:
A manual test can be executed by clicking the Run Test UI action, or you can schedule a Suite (All ⪢ Automated Test Framework (ATF) ⪢ Schedules). In either case, someone must be in attendance to select the browser and continue execution.
- Automated / unattended testing: a configured headless browser is required when hosting on the client instance, or using the ATF Test Generator and Cloud Runner.
Results and Logs
After executing a suite or test, an opportunity is given to view the results, which are also available in the Related List: Test Results or Test Suite Results. Results can also be viewed at All ⪢ Automated Test Framework (ATF) ⪢ Suite Results.
Drill into the Test Results related lists to get more specific information for a given step execution.
Debugging
The only way to see data “as it is” during test execution is with debugging. This provides an opportunity to look up data while the test is paused, before test data is rolled back.
Open a Test and right-click on a row in the Test Step related list, then select Add/Remove Breakpoint. A red dot will appear on the left side of that step row.
Now you may Debug Test. The test will pause before executing the Test Step that has the breakpoint on it. You have up to 10 minutes to check the state of the transaction.
More Information
A good foundation for using ATF: Automated Test Framework (ATF) Essentials course. There is a learning path on this page that can be followed, and a link to the Micro-Certification - Automated Test Framework test.
The main Automated Test Framework (ATF) documentation site.
Documentation on the ATF Test Generator and Cloud Runner application.
The usual SN community resources and internet searches also come into play. And who knows, an AI might occasionally get some ATF facts straight.
