I found some mistakes in PDI while ATF test is running

[DeviSriB]

Hi,

I 'm working one requirement. My requirement is for example when we are requested a catalog item like iPad Mini. So, in that catalog item have one variable that is storage. In that storage variable we have 3 options like 16GB, 64GB and 256GB. So, while we are requested that catalog item by selecting the option 1 like 16GB it goes for one approval, likewise while we are selecting option 2 like 64 GB it goes for two approvals, likewise while we are selecting option 3 like 256GB it goes for three approvals. So, for example I requested a catalog item by choosing option 1. It created one request and after that one RITM. when we are opening RITM in the related list of approval tab it shows one approver. when the approver gave the approval then only catalog task will be created. once catalog task will be closed then only next catalog task will be created. So, while I'm doing manually steps it's working fine.  Based on this requirement I have created the ATF test steps. so, while testing this ATF test steps I found some mistakes in the client runner It is taking the invalid inputs and shows the result as success. For your understanding I attached some screenshots to this found where I found mistakes in the client runner.

[Abbas_5 · Accepted solution]

Hello @DeviSriB,

The core issue is that the ATF client runner is producing incorrect results, specifically failing to recognise invalid inputs as errors and instead marking them as successes. This indicates a problem with how the client runner is interpreting the catalogue item's variable input or how it's validating the approval workflow. To resolve this, review the ATF test steps related to variable input and approval logic, ensuring they accurately reflect the expected behaviour based on the manual testing that works correctly. 

Here's a breakdown of the problem and potential solutions:

Problem:

• Client Runner Errors:

  The client runner, which is responsible for simulating user interaction and validating results within the ATF, is incorrectly reporting success when invalid inputs are provided (e.g., selecting 16GB and expecting two approvals).
• Discrepancy with Manual Testing:

  The manual testing of the catalogue item with different storage options (16GB, 64GB, 256GB) and corresponding approval workflows works as expected, indicating a gap between the manual process and the automated test. 

Potential Solutions:

1. 1. Review ATF Test Steps:

   • Variable Input: Please carefully look over the test steps that define the catalogue item's variable selection (storage options) and make sure the correct value is being passed to the client runner. Double-check the variable name and the selected option (16GB, 64GB, or 256GB) within the ATF step configuration.
   • Approval Logic: Verify the ATF steps that validate the number of approvals. If the client runner is failing to recognise an invalid number of approvals, review the validation step's configuration and ensure it correctly checks for the expected number of approvals based on the selected storage option. For example, after selecting 16GB, the ATF should validate that only one approval record exists in the RITM's related list.
   • Catalogue Task Creation: Confirm that the ATF steps correctly validate the creation and closure of catalogue tasks based on the approval process. If the client runner is not validating the correct number of tasks or their state (open/closed), review the validation steps related to catalogue task creation and closure.
2. 2. Client Script Issues:

   • Variable Manipulation: If any client scripts are manipulating the variable values or affecting the approval process based on the selected storage option, examine those scripts for any logic errors or unexpected behaviour. Ensure the scripts are correctly interacting with the variables and the approval workflow.
   • Error Handling: Check if the client scripts have proper error-handling mechanisms to catch and report invalid input scenarios. The ATF should be able to recognise these errors and fail the test accordingly.
3. 3. Approval Rules:

   • Rule Configuration: Review the approval rules configured for the catalogue item. Verify that the rules are correctly associated with the catalogue item and that the conditions for approval (based on storage option) are correctly defined.
   • Approver Assignment: Ensure that the approvers are assigned correctly based on the approval rules and the selected storage option.
4. 4. ATF Configuration:

   • Test Step Order: Verify the order of the ATF test steps. Ensure that the steps related to variable input, approval validation, and catalogue task creation are executed in the correct sequence.
   • Step Timeout: Adjust the timeout settings for the ATF test steps if necessary. If the approval process is taking longer than the default timeout, the test might be failing prematurely.
5. 5. Debugging:

   • Logging: Enable logging within the ATF and client scripts to track the execution flow and variable values. This can help pinpoint the exact location of the error.
   • Step by Step Execution: Run the ATF test steps one by one to isolate the problematic step. 

By systematically reviewing the ATF test steps, client scripts, approval rules, and ATF configuration, you should be able to identify the root cause of the issue and resolve the problem with the client runner.

If this is helpful, please hit the thumbs up button and accept the correct solution by referring to this solution in the future; it will be helpful to them.

Thanks & Regards,

Abbas Shaik