ATF provides us with some great tools to ensure our development activities don't have unintended consequences on user experience, but most people rely on executing tests manually. How many of you actually execute your test suites on every single change that's pushed into prod? Sure, you might kick off the tests for big scary changes that you're sure are going to break something, but not for the little things, and it's those little things that end up causing the big issues.
The solution is to use the wonderful 'scheduled tests' feature of ATF. This allows you to run one or more test suites on a regular basis, emailing the test results to a specified address. Scheduled tests are awesome, but they do require a 'test runner' to be active during the test window. This runner is simply an active browser session that's sitting on the atf_test_runner.do page. It sits there listening for requests to start doing stuff as part of the test, like clicking buttons and looking at the content of pages.
So how do you have a test runner ready and available for your scheduled tests? You can't leave your laptop on 24x7 with a runner window open (well, you could, but it probably wouldn't be a good idea). You could spin up a dedicated Windows or Linux desktop VM that runs the full desktop stack and a browser 24x7, but that is a big waste of resources, not to mention a time sink when the box decides to reboot every other week.
This is a simple node.js script running Puppeteer - a library that controls a headless version of Google Chrome. The script logs in to ServiceNow, navigates to the test runner page, and waits ever patiently for a scheduled test to come along and make use of its browser session. It emulates a normal Chrome session in every way, even enabling the usual screenshots to work in ATF.
The script is wrapped in Docker Compose to allow for quick deployment and easy daemonisation via systemd or your preferred management system. You can run multiple containers for different environments, if you're the type of person who likes to test everything.
So how do we set it up?
Start with the README.md file in the repository, as it will contain the most up to date information. At a high level though, you want to:
- Have some kind of server or VM sitting around
- Install the latest versions of Docker and Docker Compose (we rely on versions > 3.7 for Docker Compose)
- Pull down the repository via git or download the zip from the releases page of the repository
- Copy the .env-example file to .env, and edit the config options to suit your environment
- You should then be able to build and start the container:
docker-compose build && docker-compose up -d
Once the container is up you'll be able to confirm its success by running 'docker ps' to get a list of containers, then 'docker logs -f atf-runner-<instancename>' (replacing with the actual name). The logs should look something like this:
2019-10-08T00:01:00: PM2 log: App [atf-runner:0] starting in -fork mode-
2019-10-08T00:01:00: PM2 log: App [atf-runner:0] online
TypeError: Cannot read property 'style' of null
at https://example.service-now.com/welcome.do:298:49
at https://example.service-now.com/welcome.do:299:5
navigating to login page
not logged in
net::ERR_ABORTED [Function: url]
test runner ready
You can ignore those 'ERR_ABORTED' errors, they're coming from the browser console and are benign errors coming from ServiceNow itself.
To confirm that all worked well, you'll want to navigate to Automated Test Framework > Run > Active Scheduled Test Runners (https://instancename.service-now.com/sys_atf_agent_list.do), and confirm that your runner is in the list and has a status of 'Online'.
If that's all good then you're ready to start scheduling tests!
Some Caveats/Notes
- I've tested this in Madrid and London, but should work in other versions
- At the moment this fails silently if your credentials are wrong, or if anything else goes horrifically wrong - if the runner isn't showing up in the list then check to make sure you're logging in with the right credentials, and that the user has the correct role to run tests
- You'll probably want to set this up to start on boot - instructions for this are out of scope of this article, but some googling of terms like 'docker compose start on boot' and your OS name should get you where you need to be
- All are welcome to submit pull requests with improvements
