Flow misses some request tests

[sbettwieser]

The Problem:
When using a flow to automate testing, some requests specific tests are being missed. The missed tests are consistent and get in the way of making sure things work. Does anyone know why this happens?

Examples:
Flow Structure:

Test Results:

^ Both the status code and results exist tests are collection-level.

Login Tests:

const jsonData = pm.response.json();

pm.test("Successful login", function () {
    pm.expect(jsonData.success).to.equal(true);
});

pm.test("Correct user", function () {
    pm.expect(jsonData.user.userName.toLowerCase()).to.equal(pm.environment.get("username").toLowerCase());
});

Login Tests in Collection:

Site Id Tests:

const jsonData = pm.response.json();

pm.test("Exactly 1 result", function () {
    pm.expect(jsonData).to.be.an("array").and.have.lengthOf(1);
});

pm.test("Each result's site ID contains '" + pm.variables.get("site-id") + "'", function () {
    jsonData.forEach((item) => {
        pm.expect(item.siteId.toLowerCase()).to.include(pm.variables.get("site-id").toLowerCase());
    });
});

Site Id Tests in Collection:

[CCLS_DKR (Daniel)]

Hi,
did you ever found a way to solve this?

I’m just facing the same issue:

Test results of the executed request in the collection:

Test summary from the flow:

Just to make sure I added some logging to the console, to see whether only the test summary output would be missing. That’s not the case.

These are my first two tests, from which only the first one is executed in the flow.

var element =  pm.response.json();
console.log(element);
pm.test("Test header", function () {
   //var element = pm.response.json();
    pm.expect(element.header.step.guid).to.eql("3fbf081d-2444-4dfd-b277-f57701061c4e");
    pm.expect(element.header.businessEntity.guid).to.eql("E554D815-F958-463A-B4DD-E2EB29B29FF2");
    pm.expect(element.header.workflow.guid).to.eql("bb1ea2d1-4fab-4233-9070-be8e12f98dd7");
    pm.expect(element.header.formType.guid).to.eql("7d2b358f-54c1-49cc-8982-5c80e78f2268");
   

});
pm.test("Test tasks", function () {
    pm.expect(element.tasks.directTasks.length).to.eql(1);
    pm.expect(element.tasks.directTasks[0].user.bpsId).to.eql(pm.variables.get("userB"));
});

[sbettwieser]

No, I never found a solution. Ultimately, I decided that Flows were entirely incapable of meeting the needs of my project (for many reasons besides this one), so I haven’t made any more attempts to look for a fix.

[remcovanwingerden]

Hi, I have the same issue.

I just did a quick test to find out what could be the problem:

I just created a Request, put in 6 tests that are exactly the same but have different names. It turns out that Postman will abruptly stop after the third test when running the Request from a Flow, without any apparent reason whatsoever.

See the tests written for the Health: Success Request and 6 tests happily Passing when sent manually:

---

---

And then the same Request called from a Flow and only 3 tests ending up in the Output:

---

---

I tested it like I’ve described above because I couldn’t find out why some things in my scripts weren’t being logged to the console. Inserting some basic log statements like this:

---

console.log("1");
console.log("2");
console.log("3");
console.log("4");
console.log("5");
console.log("6");

---

into my script at the point at which it terminates unexpectedly also resulted in a log containing only a certain amount of lines and then terminating abruptly at some point, giving me the following log:

---

1
2
3

… And then nothing else to see anymore …

---

At first I suspected a user subscription tier limit for running tests in a Flow or something like that, but that’s not the case, since it clearly shows that one log statement works flawlessly and then the next just doesn’t do anything anymore.

To me it looks like some kind of timeout issue. But I can’t find anything that would suggest what’s wrong exactly.