Need your inputs re: Automated Test Framework

marilynfrancisc
Tera Contributor

‎03-13-2019 06:27 PM

Anyone here who has implemented the Automated Test Framework in their instances? Is there any licensing cost?

Your inputs and feedback are much appreciated.

  • 2,601 Views
11 REPLIES 11

Greig
ServiceNow Employee
ServiceNow Employee

‎03-13-2019 06:54 PM

Thanks Marilyn. No, ATF is a NOW platform capability that requires no additional licensing cost.

I know a few of our University customers have implemented ATF and had great reduction in regression testing and upgrades.
https://docs.servicenow.com/bundle/madrid-application-development/page/administer/auto-test-framewor...

Good luck!

Cheers,
Greig

marilynfrancisc
Tera Contributor
In response to Greig

‎03-13-2019 08:20 PM

Thanks Greig for your quick response. 

I definitely agree with you that implementing ATF will greatly impact the reduction of regression testing and upgrades. Would you mind sharing my contact information to some of your University customers whom I can reach out to discuss about their ATF Implementation?

Appreciate your help.

Cheers,

Marilyn 

0 Helpfuls

sheila_pickett
ServiceNow Employee
ServiceNow Employee
In response to marilynfrancisc

‎03-13-2019 11:44 PM

Marilyn and Greig,

The team at Harvard Business School will be speaking on this topic at K19. From their abstract (session BRE0921): "We will demonstrate how ATF reduced our regression from 40 hours to 20 minutes. This reduced delivery overhead by half while doubling our capacity to deliver value and enable our customers to do their work better."  I'll follow up with the account team to see if they are available for a discussion.

Sheila

0 Helpfuls

Terri Kouba
Tera Expert

‎03-14-2019 08:21 AM

Hi Marilyn,

The University of California, Berkeley is using the ServiceNow's Automated Test Framework and it's fantastic!  We've written about 60 of them, for different things and will be building out more because they've proven so useful.  We run them every week in our test environment, just to make sure no new code has broken anything.  And we run them during the upgrade and patch process too.

Here's when we run them in our dev instance during the upgrade/patch process:

  1. before clone of prod to sub-prod
  2. after clone and before upgrade
  3. after upgrade
  4. then we do the skip/reverts and we run the ATFs again

We are upgrading from Jakarta to London and the ATFs found things that our manual testing did not.

  1. On the projects screen, the 'score' field was removed and different 'score' fields were added.  The label of the fields are the same, so visually we thought the score field was there, but it was actually a different field.
  2. Also on the projects screen, ServiceNow added a new 'Save' button.  Again, visually the label looked the same but the code was drastically different.

In each of these cases (and more) the ATFs told us there was a difference and we were able to evaluate the changes to determine what action to take.  

We have one senior developer who created our ATF business practices and policies.  He spent a few days creating his first ATF and then handed that type of ATF over to our QA person who copied it and applied it to the rest of the tables.  Once over the learning curve, an ATF can be copied and applied to a new table in under 3 hours.  

We thrilled with the new ATF features in London and look forward it more in Madrid and beyond.

Thanks,

Terri Kouba
UC Berkeley
kouba@berkeley.edu