Hi everyone!
I just started to toy with the ATF available in Istambul version and getting some results that seem to be shadowing failed tests. I have some Server Side Validation Scripts with the next structure:
describe("My test suite:", function() {
beforeEach(function() {
// ...
});
afterEach(function() {
// ...
});
it("Test 1", function() {
// ...
});
it("Test 2", function() {
// ...
});
it("Test 3", function() {
// ...
});
});
jasmine.getEnv().execute();
Then if I code the suite like:
describe("My test suite:", function() {
beforeEach(function() {
// ...
});
afterEach(function() {
// ...
});
it("Test 1", function() {
expect(true).toBe(true);
});
it("Test 2", function() {
expect(true).toBe(true);
});
it("Test 3", function() {
expect(true).toBe(false); //this will fail
});
});
jasmine.getEnv().execute();
And run the test, SNow informs correctly that there was an error:
However, if I code my suite like the following (test failing in the middle instead of in the last place):
describe("My test suite:", function() {
beforeEach(function() {
// ...
});
afterEach(function() {
// ...
});
it("Test 1", function() {
expect(true).toBe(true);
});
it("Test 2", function() {
expect(true).toBe(false); //this will fail
});
it("Test 3", function() {
expect(true).toBe(true);
});
});
jasmine.getEnv().execute();
And run the test, SNow shadows the error status, although it properly registered the number of successful tests:
I am no Jasmine expert at all, but it seems to me like SNow is showing the final result of the suite based only on the last test placed in it. Could this be a bug in the brand-new SNow ATF or it is expected to use just one test per suite (one test for each registered SNow Test in ATF) to get a coherent status result? I don't know, but that wouldn't be very flexible, right?
Thanks in advance. Latest version is looking good!
Cheers,
Alber
Hi, was this found to be a bug and fixed? I am seeing the same behavior:
UPDATE: Same 3 expect statements will return proper result when they are all in the same Test.
Works:
Does NOT work:
So, it kind of appears like it isn't checking that EACH test passed. It would be much better if the second case works, as it makes the test failures easier to read and pinpoint.
At least I'm not crazy as well...
I "fixed" it by doing a business rule. Basically, when the output contain ":: failed" to force the status to be "Failure". See screen shots below.
