ATF Auto-Generated Tests' behavior

[Angelica G]

I've used the Automated Test Framework's new "Auto-generate tests" feature a few times now, and just got to run it again to generate a new suite after our Xanadu upgrade.

I have noticed that the generated tests do not follow the "best practice" of creating and impersonating valid users for testing. Instead, it is using existing accounts, including admins, and I'm not understanding if that's intentional. If it is, I would love to understand the generated tests more and why they generate what they do, and how I can possibly create better Presets for the generator to use, if necessary.

Any other QA's or SN admins who have had success with the tool so far, or maybe similar concerns?

[Josh Pirozzi]

Hi @Angelica G,

You're right that, if you build/develop a Test on your own, best practice is to have Test Steps where you create a new User Account and impersonate them where necessary in your Test. 

My thought with the Auto-Generated ATF Tests is that they aren't necessarily reusable but should be pulled ahead of when you're going to need them (IE: 1-2 days before you patch your non-prod Instance). Doing this will ensure you have the latest updates/modifications on each Form/Workflow in your Tests.

This is just my thought and understanding in using the Auto-Generated Tests a few times.

Hope this helps!

-Josh

[Brian Bouchard]

I personally find the generated tests are not best practice, as you mentioned, but also are often complicated and inflated, meaning they test the same thing multiple times throughout all the tests that are generated. Additionally, because it's all automated people tend to not understand exactly what they are testing.  For all of these reasons, they are difficult to maintain, and make them a poor choice for everyday use.

Josh's scenario is a good one. If you want to create a bunch of tests that you know will work , run them, then make a major change (patch, upgrade, etc), run them again and verify everything still works...  it's a good "down and dirty" solution.

Also, if you are new to ATF and just trying to get something started quickly, you can generate a small sample set of tests and then apply best practices to them, remove the steps that are unnecessarily tested in all generated tests, add some notes to document what they are actually testing, etc...  If you do this, you can use the power of the automated generation and then have truly reusable tests that you can effectively run on a regular basis and act on the results when needed.