(Knowledge Sharing) - Stop Doing Manual Regression Testing Before Every ServiceNow Upgrade

[Chanuka]

Every ServiceNow team hits this moment at some point.

A new family release is announced. Maybe you are planning to move to Australia. Maybe it is just a patch.

Either way, the same question comes up in the team chat or the standup.

"How do we make sure nothing breaks?"

And the answer is usually the same. Someone creates a spreadsheet. Business users get pulled in. Developers start clicking through forms, catalog items, and approval flows one by one.

It gets done. But every time, it takes longer than it should.

So what is ATF and why does it matter

ATF - Automated Test Framework  is not something you install from outside. It is built into the Servicenow Platform itself.

That is actually important. A lot of teams try external browser automation tools first. The problem is ServiceNow's interface changes across releases. When that happens, those external scripts break and someone has to fix them. Because ATF is part of the platform, it generally handles those changes better than external solutions.

ATF supports both server-side and client-side test steps. That means your team can validate business logic, form behavior, and common user interactions — all without relying entirely on external browser automation.

It is not a perfect tool. But for regression testing inside ServiceNow, it is the right starting point.

Why manual testing becomes the problem over time

Early in a ServiceNow implementation, manual regression is manageable. You have maybe 10 or 20 things to check. It takes a day or two.

Then the implementation grows.

Suddenly you have Client Scripts, UI Policies, Business Rules, Flow Designer flows, custom catalog items, integrations, and maybe a couple of custom applications. Each one needs validation before every upgrade.

And people test differently. One person checks everything carefully. Another misses something small. You only find out something broke after it reaches production.

That is the part that is hard to explain to management — not that testing happened, but that the wrong things got tested.

What actually changes when you use ATF

The basic idea is simple. You build tests once and reuse them every upgrade cycle.

When something fails, ATF gives you the execution details — which test step failed, what was expected, what actually happened. That is much faster than getting a message from a business user saying something feels broken and then trying to figure out where to start.

Test Suites let you group related tests together. You can schedule them to run during off-hours so they are not disrupting anyone while they run.

One scenario worth thinking about

Imagine you have test cases covering your Incident Management flow, your Change process, and your main catalog items.

Before upgrading your sub-production instance, you run the relevant ATF test suites. Instead of spending days clicking through forms manually, the tests run automatically, and your team can review the results the next morning. 

Some tests fail. You know exactly which steps. Your team focuses only on those areas.

Everything else is already confirmed. You move forward with the upgrade faster and with more confidence than before.

A few things ATF is not

ATF is not for load testing or performance testing. It is not for testing out-of-the-box ServiceNow functionality that your team has not customized.

ServiceNow also recommends keeping ATF in non-production environments. Do not run it against production.

And it does not replace UAT or exploratory testing. Those still matter. ATF handles the repetitive parts so your team can focus energy on the things that actually need human judgment.

One mistake worth avoiding

A lot of teams build ATF tests right before an upgrade, get through it, and then stop maintaining them.

Six months later those tests are outdated and nobody trusts them anymore.

The teams that get real value from ATF treat it like any other part of the codebase. When a new flow gets built or a catalog item changes, the test gets updated too. Over time that builds something genuinely useful — a regression suite you can actually rely on.

Where to start

Start with the processes your organization cannot afford to break. Not everything at once.

Build tests that are focused and modular. Group them into suites. Keep them updated as things change.

If your team is still running full manual regression before every upgrade, ATF is worth looking into seriously.

I wanted to share this thought with the community. What are your thoughts on this? How does your team manage ATF maintenance over time? 

Best Regards,

Chanuka.