Wait for data to exist - (Waiting for database/cloud to return the latest version of the data)

[postman-paul540 (PostmanPaul)]

Hi all,

I’ve been away from Postman for a year, so forgive me if this action is easily supported/ been asked before…

Basically, I have a POST/PUT request that creates data.
I am then using a GET to confirm the data entry exists and all the relevant data is there.
However, sometimes the data is not present when calling the GET straight after the previous request.
This is more prevalent in the runner as the data entry isn’t available at that time.
I tried getting around this using a ‘dirty’ wait/sleep.

https://postman-echo.com/delay/3

But this isn’t ideal/efficient/stable for the obvious reasons.

Here’s how I’m handling it:

const body = JSON.parse(responseBody);


// FYI - At the start of the test run in a previous request, I create an enviornment variable called counter and set it to 0
// And since Postman converts all env variables to string I had to parse int it.
if(parseInt(pm.environment.get("counter")) < 10) {
    pm.test("Expects a 200 response code", function() {
        pm.expect(responseCode.code).to.equal(200);
    });
    
    pm.test("Value of 'total_results' is greater than 0", function() {
        pm.expect(body.total_results).to.be.greaterThan(0); 
    });
    
    pm.test("There is at least 1 customer_return item", function() {
        pm.expect(body.data.customer_return.items).to.have.a.lengthOf.greaterThan(0); 
    });
    
    pm.test("There is no tracking reference against the customer", function() {
        const customerReturns = body.data.customer_return.items;
        const customerOrder = customerReturns.find(function(item) {
        	// In the previous POST request, I have a customerNumber env variable that I use.
          return item.customer_number === pm.environment.get("customerNumber");
        });
        // Wait for the entry to exist...
        if(customerOrder === undefined) {
        	// I take the env variable and turn it into a 'regular' variable for ease of use.
            let counter = parseInt(pm.environment.get("counter"));
            // Add 1 to the counter
            counter == counter ++;
            // Replace the env variable
            pm.environment.set("counter", counter);
            // Hit the same request again and hope for the record to exist.
            postman.setNextRequest("Search [200] - Confirms No Tracking Number");
            // Bear in the mind, the counter value is now 1, not 0.
            // After the request has been hit 9 times and the desired data is still not present, I fail the test.
            // The condition at the top of the test will not be fulfilled so it'll fail and go to the 'else' at the bottom.
            // Not using a counter would constantly spam the request until the data is there (assuming it turns up!)
        } else {
            // some tests go here once the record exists. Woo!
        }
    });
} else {
    console.log("You've looped too many times!");
    pm.test("Polled for data but customer record didn't exist", function() {
    	// I use this to have something marked as a failure in the test so the user knows what's happened.
        pm.expect(true).to.be.false;
    });
}

Would really like some feedback on this and to see how other users deal with this scenario.

[connor.sadler]

That is similar to what we ended up with
We also included a call to the ‘delay’ endpoint to avoid a tight retry loop
However we encountered problems if someone else duplicates that request or changes it’s name.
It’s not ideal.

[postman-paul540 (PostmanPaul)]

Yep, it’s pretty messy - especially having to make all my request names unique for postman.setNextRequest();
I’m glad to know others have done this a similar way to what I have shown and I haven’t just missed a feature xD

[callarman]

It would be so nice if there some sort of native support for explicit waits. It would take my postman testing to another level.

[RoBeaToZ]

Very nice way to handle those kinds of problems, this helped me a lot!

Yep, it’s pretty messy - especially having to make all my request names unique for postman.setNextRequest();

I prefer to use

postman.setNextRequest(pm.info.requestId);

instead of

postman.setNextRequest("Search [200] - Confirms No Tracking Number");

to get rid of keeping request names unique.

I have a small suggestion for

pm.test("Polled for data but customer record didn't exist", function() {
	// I use this to have something marked as a failure in the test so the user knows what's happened.
    pm.expect(true).to.be.false;
});

I use

pm.test("Polled for data but customer record didn't exist", function() {
    throw new Error()
});

It would be so nice if there some sort of native support for explicit waits. It would take my postman testing to another level.

I wrote a small copyable script as a workaround for the leak of native support:

// Retry constants
const retriesCountKey = "collection-xyz-retriesCountKey"
const retryDelayMillis = 1000
const maxRetries = 10

// Test constants
const jsonResponse = pm.response.json();
const expectedLength = 50;

// Preconditions
if (jsonResponse.length !== expectedLength) {
    console.log("Found " + jsonResponse.length + " but expected " + expectedLength);
    console.log("Retry request in " + retryDelayMillis + "ms");
    retryRequestWithDelay();
    return;
}

// Assertions
pm.test("Status code is 200", function () {
    pm.response.to.have.status(200);
});

// Retry functions
function retryRequestWithDelay() {

    if (getRetriesCount() > maxRetries) {

        pm.test(
            "maxRetries of " + maxRetries + " reached for retriesCountKey" + retriesCountKey + ", stop retrying",
            function () {
                throw new Error();
            });
    } else {
        incrementRetriesCount();
        setTimeout(() => {}, retryDelayMillis)
        postman.setNextRequest(pm.info.requestId);
    }
}

function getRetriesCount() {
    return pm.collectionVariables.has(retriesCountKey) && pm.collectionVariables.get(retriesCountKey) || 0;
}

function incrementRetriesCount() {
    pm.collectionVariables.set(retriesCountKey, getRetriesCount() + 1);
}

PS: I write my request wide variables to

pm.collectionVariables

For more information see https://learning.postman.com/docs/postman/variables-and-environments/variables/#variable-scopes to use the scope you need.