|
Tool considered for automation
|
Parameters
|
Pros
|
Cons
|
|
Automated Test framework- Istanbul version
|
- License cost
- Scripting language used
- Script execution speed
- Framework used
- Browser support
- Functional testing, Regression testing
- UI testing
- Screenshots capture/Results/Reporting mechanism
- Support model
- Ease of Integration with other tools
|
- Out-of-box application, can be used without consuming an extra fulfiller license on the Service-Now platform
- The Automated Test framework application, being out-of-the Box ( OOB ) it is being supported by Service-Now vendor as a part of the ongoing support model. It is available on the Instances starting with Istanbul release and it will be supported as a part of future Upgrade releases
- Task like opening a form, populating field values or hitting over a link on the page can be achieved without scripting, using form side configuration. Therefore simpler tasks like this can be developed without the need of a developer. This helps greatly reduce the customization effort and dependency on the build team
- Screenshots saved and attached automatically to test result after completion of each test step.
- "Test Logs" are also saved for the whole test transactions.
- Output of each transaction is saved under "Step Results".
- Test Result gets cleaned up after 30 days, default. This can also be altered in the table "sys_auto_flush". If we want to retains a specific test result then we need to check "Retain Indefinitely" check box in the test result form.
- The test framework automatically tracks and deletes any data created by running tests, automatically taking care of rolling back changes after testing. Detail logs of the test suite is available in Test Result.
- Test can be executed on a different window ( by hitting the “click here” ) which gives us a broader view of the task being executed in the UI Test Runner
- This application works on both UI 15 and on UI16
- Can be easily Integrated with other testing tools or project management tools
- The test suites can be easily executed across multiple browser versions (IE, Chrome, Mozilla, Edge etc.)
- This application is based on automated test framework (Behavioral driven)
|
- UI testing cannot be performed
- If in case multiple test suites are executed, they will get queued up. There is a time limit of 600 seconds (10 minutes), if in case the preceding test case fails to get executed within 600 seconds (10 minutes), then the next test case in the queue will get skipped/aborted
- Mandatory field validations are not done for the form side validations
- Jasmine scripting is required to automate complex Integration and non-consenting test suites on server side, knowledge of jasmine scripting is required
|
|
Selenium
|
- Selenium is an Open source software and is free
- User and community professional support is available
- Browser compatibility testing can be performed
- Test driven development framework is used
- UI testing, functional testing and regression testing can be performed
- It supports various testing frameworks, such as, JUnit, PHPUnit, TestNg, etc.
|
- Knowledge of Java, Python, Ruby, Php, Perl scripting is required to build and automate test cases
- Integrations with other tools is possible via Jenkins, Maven, Ant etc. which will involve additional effort and Knowledge
- Performance issues encountered in execution of multiple test suites at one go, or in other words, Selenium can only run one test at a time. Therefore, if there are 100 tests that need to be to executed, user will have to wait until each test finishes before the next one can begin
- If there is only one computer monitor, no other task can’t be done while the test suite is running
- If in case the scripts need to be executed on multiple browsers then they should be written using Selenium IDE and used with Selenium core or RC, else the scripts will only work on Mozilla
- There are no options available for image verification.
|
|
HP UFT (Unified functional testing)/ HP QTP (Quick Test Pro)
|
- Features available on Navigation, results validation and reports etc.
- Easy to integrate QTP with other supporting components
- QTP is integrated with Quality Center so different high quality dashboards can be configured
- Different type & level of support is available in case of any technical issues
- UI testing, functional testing and regression testing can be performed
- Test driven development framework is used
|
- Knowledge of VB script is required to build and automate test cases
- Additional license cost is involved, since it is entirely dependent on volume and the type of licenses
- The execution time is relatively higher as it takes the entire armor along pushing load on CPU & RAM.
- UFT/QTP primarily caters to windows environment and limited number of browsers support
- Integrations with other tools is possible via Jenkins, Maven, Ant etc. which will involve additional effort and Knowledge
- No screen shots are captured while test suites are executed
|
|
TOSCA
|
|
- Intuitive UI
- Compatible with lot of applications
- Cross Browser support
- In built Tosca Test framework suite is fully compatible with other testing solutions and application Lifecycle Management products
- No scripting involved and even non-programmers can write test cases
- Professional support is available
|
- Test Case exports to printer, excel or Txt format are not customizable
- Performance issues observed during execution
- For complex test scenarios, Java coding might be required which will require additional effort and Knowledge
|
