Hi All,
So I wanted two test behaviors which I believe postman/newman currently does not support (If I am wrong, it’d be great if you could point me in the right direction!):
- Retry on failure
- Exit out on the current iteration if a test failure is encountered
My current approach is to use my own function wrapper (which I have defined at a collection level pre-request script) which detects when to retry and when to fail out, and I pass it around my tests to the pm.test function.
So my tests currently look like
let utils = eval(pm.globals.get(“utils”));
let test = function() {
pm.expect …
}
pm.test(“test stuff”, utils.wrapperFunc(test));
(the wrapperFunc essentially wraps any error thrown from the “test” function, if it sees that it is not applicable for retry, it throws back out the same error)
This wrapper function works mostly as expected, except in the html reports/json, it seems like that the pm.test no longer understands when it fails or not. (inside the json reporter, it sees that there are test failures, but they are no longer associated with each request).
Can someone help me better understand how pm.test associates a test with a request / is there a better approach to the way I am doing things?
Thanks!
hi @totoro.bloop
I’m not quite sure I understand exactly what you’re doing, in particular not sure how you’ve implemented your retry mechanism, and that’s ok.
I do similar things and used postman.setNextRequest() in the test script. ie. when I receive a response that makes me want to retry, I then set the next request back to itself. (Being careful to avoid infinite loops.) eg.
The collection runner (and elsewhere too) then looks as if you’ve executed several separate requests one after the other, each with its own test results. eg.
If you’re trying to keep it DRY (Dont Repeat Yourself) then you could use this in the collection test script when you want to retry the same request.
postman.setNextRequest(pm.info.requestName);
When you’re ready to finish the current iteration just use
postman.setNextRequest(null);
Alternatively, you could try and run your collection with the newman module in a node.js script and then debug the actual reporter node module in Chrome. Sounds a bit tricky/whacky, but I managed to do it a while ago, didn’t turn out to be too hard. It helped me solve a problem at the time, and was a kinda cool way to learn how the reporters work. I can dig out the details of how to do that if you’re interested. You can set breakpoints on the Newman events that the reporter listens for. (but this will likely head you down the track of writing your own reporter module)
