Automated Test Framework Part 3: Building Blocks

Here's the series so far:

1. Automated Test Framework Part 1: First Impression
2. Automated Test Framework Part 2: Form Submit
3. Automated Test Framework Part 3: Building Blocks

Hi,

Related list validation(Edit/New button visibility onclick() action based on role)

can it be done using ATF?

Thank you.

You'll need to create a Test Step Configuration for that. You can look at some OOB Test Step Configs, for example, UI Action Visibility / Click a UI Action, and make a copy and modify them.

How about creating a load test with ATF?

Hi John,

I just wanted to give you a shout out and thank you for the feedback and review of the ATF. It's extremely helpful and provides a great overview and hand-on experience of what to expect on initial view.

Thanks for sharing - appreciated.

Robbie

Thanks Robbie - glad to hear that.

John

What kind of load testing do you have in mind? FYI, ATF can run tests only in series (one test at a time).

Have you figured out a way to determine if a drop-down doesn't contain certain choices? It seems that we can only test for the presence of a value but not the absence of one. For instance, the ability to set the state of an incident to "Closed" is restricted by removing the option through a client script. We would like to test that only the values we expect to see are available but I haven't found a good way to test this through automation.

Also, what about validating field messages?

Let me try to provide some clues to get you going.

Prerequisite: you'll need to have a good understanding of querySelector (aka CSS Selector).

Navigate to Automated Test Framework > Administration > Step Configurations and open "UI Action Visibility". Then inspect Step execution script and scroll to about halfway until you see the following function:

function checkVisibility(actionId) {

   var queryString = "[gsft_id='" + actionId + "']";

   var actionIdExist = testFrameWindow.document.querySelector(queryString);


   if (actionIdExist == null)

       return false;


   if (!isVisible(actionIdExist))

       return false;


   return true;

}

You can copy this to create your own Step Configuration, but with the following:

var valueExist = testFrameWindow.document.querySelector(queryString);

Here, replace "queryString" with the CSS Selector for the "Closed" select option element in the DOM (View Source in the browser will show you what to use). Then you can use it to test the presence or absence of the dropdown value in the field.

You can take a similar approach for testing field messages.

I'd recommend writing a generic function that can test the presence/absence of any DOM element and using it in a reusable Test Step Config.

ps. sorry, I always like using Syntax Highlighting for JavaScript but the editor isn't giving me that option today, somehow. I had to manually insert the <pre> tags to get what you see here.

Hi John,

Thank you for the response. I was attempting to follow the path you provided however I hit a few road blocks. For starters, the "Step Environment" field will only allow me to select "Server - Independent' and not "UI'. Reviewing the dictionary entry for this field shows that the filter is set to not show the "UI" option. If I remove that filter so that I can select it, I receive an error stating that I need the "maint" role to be able to create a test step with that step environment. I cannot add this role to my user account even with the 'security_admin' role and the 'Elevate Roles' option enabled.

Were you able to copy the "UI Visibility" Step Configuration and modify it accordingly?

Hi Todd - there are a lot of hoops to jump through to get this working; there must be a good reason for that, so please proceed with caution at your own risk. You can create your own Test Step Config by cloning an OOB record using a Background Script similar to below:

var grOld = new GlideRecord('sys_atf_step_config');

grOld.get('d8fdf5e10b1022009cfdc71437673adc');   // OOB "UI Action Visibility"

var grNew = new GlideRecord('sys_atf_step_config');

grNew.newRecord();

for (var f in grOld) if (f != 'sys_id') grNew[f] = grOld[f];

grNew.name += ' Clone';

grNew.sys_policy = 'NULL';   // make it Unprotected

grNew.setWorkflow(false);   // don't run Business Rule "Check step environment"

grNew.insert();

Please note that this doesn't clone Related Lists. Also, when you try to update this in the form after making changes, the "Check step environment" business rule will prevent your from doing so, which means you can update the record only using Background Script with .setWorkflow(false).

Hi Todd - I recently learned that the inability to create custom Form Step Configurations is by design, as I suspected. The reason I was told is because form/UI/browser tests tend to be "brittle", that is, they tend to break easily with changes in UI (by switching to a new UI, for example).

Thank you for the follow up John!

Is there support in the ATF for unhappy path testing? i.e. make sure a message appears when you try to submit a form with an invalid value.

As mentioned in the above thread, UI tests, including "unhappy path" or negative tests, are limited to the available OOB Form Step Configurations, since we can't create our own. In Istanbul, I'm not aware of a way to test what you're describing.

Hi Sagar,

Any chance you created the Test Step Configuration for creation of a new record from a   Related List?

Use Case:   On the Problem Form, click the "New" UI Action on the Problem Tasks Related List

Thank you,

John

Thanks for this nice blog John Chun.

Can you please share how ServiceNow rolls back all the changes it made after executing tests? I.e., Data Cleanup. (e.g. cleaning up of incidents/tasks etc which are created as part of test)

Have got two important considerations before executing tests.

1. Does SN rolls back only the data that was created during the tests? Or rolls back all the data created in that session?

2. What if another user creates some legitimate data (in another session) at the same time while tests are running? Is this data also going to be rolled back?

3. When does the roll back happens? Is it after each test run or when the full test suite completes execution?

4. Is there any mechanism to create and restore save points before and after executing tests?

5. Is there any guidance/recommendation on setting up of large data before running a large test suite?

ATF in Jakarta unleashes a lot of ability to automate tests, but while considering automation at a medium to large scale, need to consider all the above points before going through this route. I have been using Selenium to automate tests, but this ATF seems to be much quicker and easier for any user to automate.

Thanks for your time,

-Praveen

Can we test UI Pages using ATF.

can we set sequnence of execution in multiple child test suites which are at same hierarchy level in a parent test suite.