ATF Test Generator - in which instance do you generate the tests?

[pavla]

Hi, we would like to use the ATF Test Generator to generate automated tests. 

Our concern is that if we generate the tests in the testing instance, the quality of tests will be poor and the tests will be inaccurate because the testing instance does not reflect the actual activity happening in the production instance.

A possible solution is to generate tests in a different instance and transfer the tests to the instance where we want to execute them.

Question 1: Generating tests in production seems risky. I suppose, that the tests are actually executed during the test generation. So it could trigger unintentionally some processes and create temporarily dummy tickets. Am I right ?

Question 2: So cloning production instance to a pre-production instance seems to be the best solution. But I am wondering if it is actually useful if the clone doesn't contain transactional data.

[Giles Lewis · Accepted solution]

Hi Pavla.

You generate your tests in the testing instance.

You stated above that "testing instance does not reflect the actual activity happening in the production instance".  This suggests that you have not cloned recently; so the first thing to do is to clone over your testing instance. You should clone over your testing instance whenever the data begins to deviate significantly from production.  I recommend that you clone over your testing instance at least once every six months, but not more often than once/month. 

If you are cloning over your testing instance monthly, then how do you keep your tests from getting erased?  Sandeep makes several suggestions:

• Use data preservers to keep them from getting overwritten during the clone
• Use Update Sets to back up and restore your tests

Sometimes organizations will store a copy of their tests in production just to make sure that the tests are preserved during a clone. However, you should never create tests in production and you should never execute tests in production.

I have found "apps" to be a helpful way to manage ATF tests, especially when you are working in the global scope.  You can go to sys_app.list and create a new custom app in the global scope for the sole purpose of holding ATF test.  In other words, the app does not contain any tables or code: only ATF tests. You can use the "Move to Application" UI Action to move ATF Suites, Tests and Steps into or out of the app. Just remember that when you move a Test to an app it does not automatically move the Steps. However, if you copy a Test (per the KBA referenced above), then it should automatically copy the Steps. Once you are sure that the app contains all the components of your Test Suite, you can export the whole thing as an Update Set. 

[Community Alums]

Hi @pavla ,

You are pointing into the correct direction!!

Running in PROD is Strict No No, it is not recommended to do so. By default, the system property that is used to run automated tests is disabled to prevent you from accidentally running these tests on a production system. To avoid data corruption or an outage, run tests only on development, test, and other non-production instances.

• It is recommended to transfer test using update set, using preservers during the clone process is untested and therefore not recommended.
• Promoting ATF to production instances is an option as well since clients would be cloning over sub production, but remember run tests only on development, test, and other non-production instances.
• You may need to check the sys_variable_value.value field to make sure that the reference it is expecting exist on the target instance, otherwise a valid reference would need to be provide before the test can pass. 

NOTE: For existing ATF Tests , you may refer to KB0825608-Capture an existing ATF test in a new update set

[pavla]

Hi Sandeep, thanks for answer. 

I know that running tests in prod is not recommended. I was just wondering if we could only generate the tests there and transfer them to the testing instance. Or should we rather clone an instance and generate the tests in the cloned instance? I simply dont know where to generate the tests to get the most accurate tests possible.

[Community Alums · Accepted solution]

Hi @pavla ,

It's never transferred from Prod to Lower instances.

You need to clone Prod to Non-prod instance and  generate the tests.

[Giles Lewis · Accepted solution]

Hi Pavla.

You generate your tests in the testing instance.

You stated above that "testing instance does not reflect the actual activity happening in the production instance".  This suggests that you have not cloned recently; so the first thing to do is to clone over your testing instance. You should clone over your testing instance whenever the data begins to deviate significantly from production.  I recommend that you clone over your testing instance at least once every six months, but not more often than once/month. 

If you are cloning over your testing instance monthly, then how do you keep your tests from getting erased?  Sandeep makes several suggestions:

• Use data preservers to keep them from getting overwritten during the clone
• Use Update Sets to back up and restore your tests

Sometimes organizations will store a copy of their tests in production just to make sure that the tests are preserved during a clone. However, you should never create tests in production and you should never execute tests in production.

I have found "apps" to be a helpful way to manage ATF tests, especially when you are working in the global scope.  You can go to sys_app.list and create a new custom app in the global scope for the sole purpose of holding ATF test.  In other words, the app does not contain any tables or code: only ATF tests. You can use the "Move to Application" UI Action to move ATF Suites, Tests and Steps into or out of the app. Just remember that when you move a Test to an app it does not automatically move the Steps. However, if you copy a Test (per the KBA referenced above), then it should automatically copy the Steps. Once you are sure that the app contains all the components of your Test Suite, you can export the whole thing as an Update Set.