Unable to run server side script in ATF reusable tests

[Terence N]

Hi,

I am trying out reusable tests in ATF to do test environment setup, but I kept encountering the "The step result was not set correctly, all steps must be set to success or failed" error while running server side scripts. I'm pretty sure the configurations are correct.

I also tried running the exact same code in my main test and it managed to pass.

Is this a bug, or am I missing something here? Does anyone face the same issue?

[AhsanM · Accepted solution]

I have seen this exact issue before. It is not a bug, it is a subtle but important difference in how ATF executes server side scripts in reusable tests versus main tests.

Looking at your screenshots, the same script passes in your main test (Task Creation) but fails with "step result was not set correctly" in the reusable test (Business Application Setup). This is a classic sign that the stepResult is not being set inside the reusable test context.

The most likely cause:

When a server side script step runs inside a reusable test, the execution context is slightly different. The script needs to explicitly set stepResult to either success or failed in ALL code paths including any conditions or early returns. If any code path exits without calling stepResult.setSuccess() or stepResult.setFailed(), the reusable test throws this error even if the main test handles it fine.

Looking at your script around lines 84 to 94, check if there are any code paths that could exit without hitting either setSuccess() or setFailed(). For example if the try block exits unexpectedly before line 87, the result never gets set.

Fix to try:

Add a default fallback at the very end of your script before the closing bracket:

(function(outputs, steps, params, stepResult, assertEqual) {
    try {
        stepResult.setOutputMessage("Record found");
        stepResult.setSuccess();
        return;
    } catch (error) {
        stepResult.setOutputMessage("Record not found: " + error);
        stepResult.setFailed();
        return;
    }
    
    // Safety fallback
    stepResult.setOutputMessage("Unexpected exit");
    stepResult.setFailed();

})(outputs, steps, params, stepResult, assertEqual);

Also worth checking:

Make sure the reusable test step config is set to "Run Server Side Script" and not a different step type, which your screenshots confirm looks correct.

Hope this resolves it!

[Terence N]

Hi Ahsan,

Thanks for the detailed explanation and suggestion. I retried it today and the problem disappeared without any changes, but I now face another problem with the server side script in the main test not able to run after a reusable test step. I will raise this in another post.

I just want to briefly comment about your suggested fallback stepResult. In my code, the try-catch block is supposed to handle all scenarios. If something goes wrong in the try block, the catch block should still set the stepResult to failed. The fallback stepResult is unnecessary. Even the linter says it's unreachable code. If we really need that setResult, I fear we may have a deeper problem with the engine. My inkling is that there may be caching somewhere that kept using an older version of the script, that's why the problem disappeared on its own. But that sounds absurd as well.

Anyway, thanks again for the help. Have a nice day ahead.