Hi,
in our test environment, unfortunately quite some requests are failing from time to time, due to timeouts, environment not being available, you-name-it, so I want to give that requests a retry, especially when running the collections in Jenkins via newman.
So I came up with a bit of code, that repeats waits for a certain amount of time, before it does a retry of the request. The number of retries can also be set.
var expectedHttpStatus = 200;
var maxNumberOfTries = 3;
var sleepBetweenTries = 5000;
if (!pm.environment.get("collection_tries")) {
pm.environment.set("collection_tries", 1);
}
if ((pm.response.code != expectedHttpStatus) && (pm.environment.get("collection_tries") < maxNumberOfTries)) {
var tries = parseInt(pm.environment.get("collection_tries"), 10);
pm.environment.set("collection_tries", tries + 1);
setTimeout(function() {}, sleepBetweenTries);
postman.setNextRequest(request.name);
} else {
pm.environment.unset("collection_tries");
pm.test("Status code is " + expectedHttpStatus, function () {
pm.response.to.have.status(expectedHttpStatus);
});
// more tests here...
}
Hope it might be of use for someone!
Cheers,
Christian
14 Likes
Nice!
I guess this could be put inside the collection window too? So it’s applicable for all requests in the collection (assuming you’re not doing any negative testing)
I do something kinda similar to test an endpoint that’s using ElasticSearch.
So I create an entry (In my case it’s a customer return in the DB) I then try to use our “search” endpoint to confirm the customer return can be searched, but you have to wait 5-10 seconds for the entry to exist in Elastic, so two solutions:
- Hardcode a wait for 10 seconds or so
- Keep hitting the search request x times until the entry is shown, if the entry doesn’t appear after x times then consider the test a failure.
I obviously prefer the second solution due to obvious reasons 
@chrissbaumann This is a good example to publish as a template in Postman!
3 Likes
Did you set up a new instance of Elasticsearch and send it a bunch of test data to query against via Postman or was that part handled differently?
Two small but life quality improvements from me.
- Small improvement to
You can use pm.environment.has instead of that if( not…).
Standalone function code
// HotFix Retry function
const statuscode = pm.collectionVariables.get("StatusCode")
if(pm.response.code != statuscode){
let retries = pm.collectionVariables.has("retries") && pm.collectionVariables.get("retries") || 0;
let attemptRetry = pm.collectionVariables.get("attemptRetry");
console.warn("retries used "+retries+" of " + attemptRetry);
// time is in ms
setTimeout(function(){}, 200);
if(retries < attemptRetry){ // Main condtion
postman.setNextRequest(pm.info.requestName);
pm.collectionVariables.set("retries", ++retries);
console.warn("Increase retries "+retries);
}
else{
// Reset variable
pm.collectionVariables.set("retries", 0);
console.warn("Reset retries. Failed after " + retries);
// OR remove it
//pm.variables.unset("retries");
postman.setNextRequest("Next_request");
}
}
else{
pm.collectionVariables.set("retries", 0);
console.warn("Reset retries on request Success "+ pm.collectionVariables.get("retries"));
postman.setNextRequest("Next_request");
}
Similar but with variables set in collection not in script.
- attemptRetry
- retries
- statusCode
I Use setNextRequest as Newman sometimes stopped execution after retry ended.
- As a final approach you can Pack it as a global function:
Collection predefine helpers
pm.globals.set('helper', function loadhelper() {
let helper = {};
helper.retry= (statuscode, maxRetries, Next_request, sleep=200) => {
if(pm.response.code != statuscode){
let retries = pm.collectionVariables.has("retries") && pm.collectionVariables.get("retries") || 0;
console.warn("retries used "+retries+" of " + maxRetries);
// time is in ms
setTimeout(function(){}, sleep);
if(retries < maxRetries){ // Main condtion
postman.setNextRequest(pm.info.requestName);
pm.collectionVariables.set("retries", ++retries);
console.warn("Increase retries "+retries);
}
else{
// Reset variable
pm.collectionVariables.set("retries", 0);
console.warn("Reset retries. Failed after " + retries);
// OR remove it
//pm.variables.unset("retries");
postman.setNextRequest(Next_request);
}
}
else{
pm.collectionVariables.set("retries", 0);
console.warn("Reset retries on request Success "+ pm.collectionVariables.get("retries"));
postman.setNextRequest(Next_request);
}
};
//// and next Global helper function.
//// helper.testModelXYZ () => {};
return helper;
} + '; loadhelper();');
Two liner usage:
// Use helper function from directory
const helper = eval(pm.globals.get("helper"));
pm.test("Make some retry- Generally test name", helper.retry(200,3,"some next req", 5000));
You can store it in CollectionVariables but If you use version control mechanism it is annoying to see this variable changed in almost all PR.
Put this on some Make stuff request number one in collection.
Or on the directory level of the collection but this may be expensive as this JS will be executed too frequently in my humble opinion. You can always wrap it with pm.environment.has and skip if it is already available.
Predefined Helpers function is the way for me for large repeated part of JS code in Postman.
Take it and share it as it make retries as part of your “Swiss knife toolkit”
3 Likes
Using this function, how I can set Newman exit code 0 once one of the retries attempt has been succeeded.
The precondition as is follows:
- There is a user-flow (folder with a list of requests with a relevant tests).
- User-flow running under Newman + htmlextra ci/cd pipeline
- User-flow running on test environments which is has 3 party dependencies (services, networking etc)
We’ve the following issues:
Network timeouts, socket hang up.
Which why we are using retry mechanism
that allows us to repeat the requests once test got status code other than 200 as shown below
So eventually the user-flow itself pass as you see at the print screen
(the second retry succeded) but Newman return fail on such user-flow
once we got at least one timeout.
The question is: How i can overcome such issue
Once the retry succeeded i would like newman will return Success (exit code 0)
@chrissbaumann
@michal.robaszewski.j
Hey @StanislavMasarsky
Can you also show the script you’re using inside the collection?
The script above is what @chrissbaumann was using in his context but I’m guessing that yours will have slight variations and not copy and pasted into your Collection 
The final report image shows the after effect of that running and doesn’t really give much information about the workflow inside the collection.
I’m using the same retry function as above and retry itself works perfectly but as I said once one of the retry attempt doesn’t succeeded newman return exit code 1
The retry script placed under collection:
// Retry mechanism
pm.globals.set('helper', function loadhelper() {
let helper = {};
helper.retry = (statuscode, maxRetries, Next_request, sleep = 2000) => {
if (pm.response.code != statuscode) {
let retries = pm.collectionVariables.has("retries") && pm.collectionVariables.get("retries") || 0;
console.warn("retries used " + retries + " of " + maxRetries);
// time is in ms
setTimeout(function () { }, sleep);
if (retries < maxRetries) { // Main condtion
postman.setNextRequest(pm.info.requestName);
pm.collectionVariables.set("retries", ++retries);
console.warn("Increase retries " + retries);
}
else {
// Reset variable
pm.collectionVariables.set("retries", 0);
console.warn("Reset retries. Failed after " + retries);
pm.expect.fail(`This failed due to newtwork issues`);
//console.error("Failed after " + retries);
// OR remove it
//pm.variables.unset("retries");
postman.setNextRequest(null);
}
}
else {
pm.collectionVariables.set("retries", 0);
console.warn("Reset retries on request Success " + pm.collectionVariables.get("retries"));
postman.setNextRequest();
// return process.exit(0);
}
};
return helper;
} + '; loadhelper();');
and under request test tab the following test:
const helper = eval(pm.globals.get("helper"));
pm.test("Make retry ", helper.retry(200, 2, "RemoveDriverOffer", 2000));
You seem to have this postman.setNextRequest(); line which won’t do anything.
That function takes an argument, either null to stop the execution of the name/id of the next request.
@michal.robaszewski.j updated function uses the Next_request value passed into the retry function - It doesn’t look like you’re using that anywhere.
As according to the Postman documentation workflows should work as is follows:
If postman.setNextRequest() is absent in a request, the collection runner defaults to linear execution and moves to the next request
Isn’t it?
So even with the postman.setNextRequest() is empty the next request executed
and indeed it is.
This doesn’t affect on the flow execution the flow executed as expected
but Newman return exit code 1
1 Like
What’s the point in having that line in there then 
Newman itself has a number of different flags that could be used - I’m not sure what’s causing it to return that exit code but you could look into:
-x, --suppress-exit-code
Specify whether or not to override the default exit code for the current run.
--timeout <ms>
Specify the time (in milliseconds) to wait for the entire collection run to complete execution.
--timeout-request <ms>
Specify the time (in milliseconds) to wait for requests to return a response.
As you said above it’s copy&paste but even though that’ line not the point 
I’ve tried Newman CLI as well and it’s not help me.
This one: -x, --suppress-exit-code for example just override exit code forwhole pipeline flow
In my case i need yo use exit code override only if one of the retries has been succeeded
@michal.robaszewski.j can you advice please?
Hi, The variable Next_Request when using setNextRequest is crucial.
When you Break the Linear execution you are in command.
If you declare setNextRequest in script with null or empty value it will just stop execution when it hit that line using Newman. I don’t know if this is expected feature but providing variable with next request name is god enough solution in most cases.
You have to provide next request to chain it again if you want it to continue.
In next request it will just flow again with Linear logic top to bottom.
Its hard to maintain the naming of following request to be executed but with some name convention you can deal with it.
Cheers!
As according to the Postman documentation workflows should work as is follows:
If postman.setNextRequest() is absent in a request, the collection runner defaults to linear execution and moves to the next request
So even with the postman.setNextRequest() is empty the next request executed
and indeed it is.
Does it give you a different result if you add postman.setNextRequest(Next_Request); to the 2 places that @michal.robaszewski.j has it in the script you copied. He authored that script so would have more context around the execution paths and why things are in there.
This is also the incorrect way to use that function postman.setNextRequest(); so if you’re using it, you might as well use it as it’s been implemented, add null or a request name/id, or just completely remove it from the script. 
Moving past this, what did you want to see happen on the final report? Newman is telling the reporter what happened during the execution, if it sees that a request was run it will show it. The same thing will happen on the CLI output and any other reporter that gets it’s data from Newman.
The script work as expected for both ways:
postman.setNextRequest(Next_Request)
postman.setNextRequest()
and linear flow executed as well for both but Newman return exit code 1 once
one of the retries attempts has been failed
Is there a reason or an error message to suggest why it fails.
Did the script fail or the request fail?
The exit code in itself doesn’t really give anyone any clues as to why it failed.
of course all the point of the retry mechanism is to overcome such issues as timeouts and hang up
so in my case this timeouts
At the end what I need to do just some how to make Newman to ignore retry failed once at least one of the attempts succeeded
1 Like
