Async Operations!

singhsivcan · May 31, 2021, 9:18pm

I was working with a colleague and we were trying to achieve something by chaining multiple requests as a pre-step before the actual request happened.

Use case:
We wanted to call 3-4 specific endpoints in a specific order and gather all the required data before the actual request was sent out.

One way was to do it by creating multiple requests and using the collection runner. But the other way was to make use of pm.sendRequest, however, there’s no guarantee in which order multiple of pm.sendRequest would execute during the execution phase of the script.

So, I ended up creating 2 operations of the famous async library (probably will work on adding more) in the most naive way as possible, but it works so… ¯\(ツ)/¯

This is just an example to showcase what’s do-able using scripts, and how a lot of things can be achieved with chaining everything in just one request.

Reference:
See the pre-request scripts of each request in the following collection and fork the collection if you want to try it out:
https://www.postman.com/postman/workspace/postman-answers/collection/3407886-220af2f6-63f2-4d84-903d-99e6e296a8c8?ctx=documentation

Example code snippet:

// Expected output in console: 3, 1, 2
asyncSeries([
    (cb) => pm.sendRequest('https://postman-echo.com/delay/3', (err, res) => {
        console.log(res.json().delay);
        cb(err, res);
    }),
    (cb) => pm.sendRequest('https://postman-echo.com/delay/1', (err, res) => {
        console.log(res.json().delay);
        cb(err, res);
    }),
    (cb) => pm.sendRequest('https://postman-echo.com/delay/2', (err, res) => {
        console.log(res.json().delay);
        cb(err, res);
    })
], (err, res) => {
    console.log('Series operations resolved', err, res);
});

Screenshot:

sean.keegan · June 2, 2021, 7:22pm

This is awesome @singhsivcan! Thanks for putting this in a collection that I can play around on my own.

And that documentation in the Pre-request tabs?!

praveendvd · June 2, 2021, 10:02pm

@singhsivcan It looks great! just had a question is there any drawbacks for below approaches when compared to this

Approach 1

pm.sendRequest("https://postman-echo.com/delay/6", (error, res) => {

    console.log(res)

    pm.sendRequest("https://postman-echo.com/delay/1", (error, res) => {

        console.log(res)

        pm.sendRequest("https://postman-echo.com/delay/2", (error, res) => {

            console.log(res)

        })

    })

})

Approach 2:


pm.environment.set("final", [])

let promisify = function (req) {

    return new Promise((resolve, reject) => {

        pm.sendRequest(req, (error, res) => {

            setTimeout(resolve(res), 1);

        })

    });

};

async function test() {

    await promisify("https://postman-echo.com/delay/3")

    await promisify("https://postman-echo.com/delay/1")

    await promisify("https://postman-echo.com/delay/2")

}

test()

//set time out to the max response time of the 3 requests

// here 3000 or 3 sec is the highest so we are keeping 5000

setTimeout(()=>{},5000)

singhsivcan · June 2, 2021, 11:00pm

@sean.keegan Thanks, I am glad it’s useful!

@praveendvd Thanks for sharing your thoughts. My thoughts over the approaches you shared:
Approach 1: Callback hell! Yes, that’s what promises help you avoid, but currently the Postman script execution doesn’t support promises. So for lot of operations, this approach becomes a really big problem.

Approach 2: You’re kind of simulating a race situation here since again promises don’t work as you’d expect them to in the postman eco-system. I’ve personally found async library’s ways of working with callbacks pretty awesome and that’s why I prefer going with the approach that we used in the requests.
Also, since you’ve tried to simulate a race condition here, I added it as a new request in the collection itself: async.race !

praveendvd · June 3, 2021, 4:35pm

@singhsivcan Thank you so much. That’s really cool , i simplified that code by calling the cdn equalent of the async library . Really thanks for this learned something new today


// calling async library cdn function

pm.sendRequest("https://cdnjs.cloudflare.com/ajax/libs/async/3.2.0/async.js", (error, res) => {

    //initializing async

    (new Function(res.text()))();

    //create a call back function that will act as the iteratee

    //we should call callback() for the async library to work

    let requestCallbackWrapper = function (req, callback) {

        pm.sendRequest(req, (error, res) => {

            console.log(res.json())

            callback()

        })

    };

    //pass the urls as array in order

    //each item and a callback will be passed to iteratee we created

    async.eachSeries(["https://postman-echo.com/delay/3", "https://postman-echo.com/delay/5", "https://postman-echo.com/delay/1"], requestCallbackWrapper, function (err) {

        console.log(err);

    });

});

praveendvd · June 3, 2021, 8:22pm

@singhsivcan The above code works but if i try to use the same code again it will throw error cannot call eachSerise of undefined. Could you help me out i am not understanding whats wrong .

steps to reproduce:

Add the above script to both pre-request and test script
Now send the request

praveendvd · June 3, 2021, 8:48pm

using min build works:


pm.sendRequest("https://cdnjs.cloudflare.com/ajax/libs/async/3.2.0/async.min.js", (error, res) => {

    //initializing async

    eval(res.text());

    //create a call back function that will act as the iteratee

    //we should call callback() for the async library to work

    let requestCallbackWrapper = function (req, callback) {

        pm.sendRequest(req, (error, res) => {

            console.log(res.json())

            callback()

        })

    };

    //pass the urls as array in order

    //each item and a callback will be passed to iteratee we created

    this.async.eachSeries(["https://postman-echo.com/delay/3", "https://postman-echo.com/delay/5", "https://postman-echo.com/delay/1"], requestCallbackWrapper, function (err) {

        console.log(err);

    });

});

erhhung-intel · June 9, 2021, 5:41am

but currently the Postman script execution doesn’t support promises

Really? I use it to promisify pm.sendRequest() and with async/await:

Collection pre-request script:

// export common utility functions
pm.globals.set('util', String(() => ({

    // use the open interval hack to wait until async
    // operations are done before sending the request
    // in a pre-request script
    waitUntilDone(promise) {
        const wait = setInterval(() => {}, 300000);
        promise.finally(() => clearInterval(wait));
    },

    // promisified pm.sendRequest()
    sendRequest(req) {
        return new Promise(
            (resolve, reject) => pm.sendRequest(req, (err, res) => {
                if (!err && res.code / 100 < 4) return resolve(res);
                let message = `Request "${req.url||req}" failed (${res.code})`;
                if (err?.message) message += `: ${err.message}`;
                reject({message});
            }));
    },

    // load external library modules in order,
    // then return this[thisProp] or just this
    async loadModules(urls, thisProp=undefined) {
        const thisKeys = Object.keys(this);
        (await Promise.all(urls.map(this.sendRequest)))
            .forEach(res => eval(res.text()));

        const thisObj = _.omit(this, thisKeys);
        //console.log('KEYS: this', Object.keys(thisObj));
        return !thisProp && thisObj || thisObj[thisProp];
    },
})));

Main request pre-request script:

// import common utility functions
const util = eval(pm.globals.get('util'))();

async function echoDelay(secs) {
    const url = `https://postman-echo.com/delay/${secs}`;
    return (await util.sendRequest(url)).json().delay;
}

async function asyncSteps() {
    console.log(await echoDelay(3));
    console.log(await echoDelay(1));
    console.log(await echoDelay(2));

    // perform some other async file loading
    const {moment} = await util.loadModules([
        'https://momentjs.com/downloads/moment.min.js',
        'https://momentjs.com/downloads/moment-timezone-with-data.min.js',
    ]);
    const tsToDate = ts => {
        const date = new Date(ts);
        let   str  = moment(date).format('L hh:mm:ss');
        const tz   = moment.tz(date, moment.tz.guess());
        const dur  = moment.duration(date - new Date());
        return `${str} ${tz.zoneAbbr()} (${dur.humanize(true)})`;
    };
    console.log(tsToDate(1615715966000));
}
util.waitUntilDone(asyncSteps().catch(console.error));

singhsivcan · June 9, 2021, 11:30am

It doesn’t in contrast with the native behaviour of promises, you cannot halt script execution where the script will wait till all your promises resolve/reject, the script execution only halts for setTimeout/Interval(s) or pm.sendRequest function calls, and this also is one way to hack around that you shared:

erhhung-intel:

// use the open interval hack to wait until async
    // operations are done before sending the request
    // in a pre-request script
    waitUntilDone(promise) {
        const wait = setInterval(() => {}, 300000);
        promise.finally(() => clearInterval(wait));
    },

jprealini · September 30, 2021, 9:28pm

Thanks @singhsivcan ! This helped me solve a scenario I had with multiple calls… Now, I question I have is… I am trying to assert at least the status code for each of the calls (three in fact)… But when any of them fail, instead of showing a FAIL test in the Test Results, it shows the infamous “AssertionError” red sign… Is there a way to make that work correctly?

Also… let’s say in the last call of the async series I need to delete some data I created in the first call of the series… but if any of the intermediate calls’ assertions fails, the serie is not completed, thus the data never gets deleted… is there a way to force the subsequent calls to be made even if a prior one has a failing assertion?

Thanks!

twnk · September 13, 2022, 3:43pm

Hey @singhsivcan! This is fantastic, thank you Could you clarify, is there currently a licence for this code?

singhsivcan · September 13, 2022, 4:00pm

There isn’t any. Feel free to use it, however.

pauledwardsleo · September 13, 2023, 1:29am

@singhsivcan thank you for sharing your Async Operations solution Collection.

However, it appears the async.series entry in the Collection may have been inadvertently clobbered because its Pre-request Script tab currently contains only this 1 line of code:

console.log(pm.request.auth)

singhsivcan · September 13, 2023, 1:58am

Thank you for notifying me. Looks like it had been updated recently by someone. I’ve reverted it back.

pauledwardsleo · September 27, 2023, 8:00pm

@singhsivcan It appears that lodash is referenced in the async.waterfall example, but is not used.

Furthermore, it seems lodash is not actually required in any of these examples and could be refactored to use array .length instead. According to benchmark evidence .length is about 4.3x faster than lodash .size and would require less memory overhead by not loading lodash at all.

yosrixp · December 28, 2023, 5:47am

You can achieve multiple requests by using Promise.all i.e

const fetch = (req) => {
    return new Promise((resolve, reject) => {
        pm.sendRequest(req, (err, res) => {
            const resp = {};
            if (err) {
                reject({ reqId: err });
                return;
            }
            resp[req.reqId] = res.json()
            resolve(resp);
        });
    })
}

const promisify = async (requests) => {
    try {
        return await Promise.all(
            requests.map(req => fetch(req).then(res => res))
        )
    } catch (error) {
        console.log("Error", error)
    }
}

promisify([{
    reqId: 'delay3', url: 'https://127.0.0.1:4443/delay3'
},
{
    reqId: 'delay2', url: 'https://127.0.0.1:4443/delay2', method: 'POST',
    header: {
        'Accept': 'application/json',
        'Content-Type': 'application/json'
    },
    body: JSON_REQUEST
},{
    reqId: 'delay1', url: 'https://127.0.0.1:4443/delay1'
}])
    .then(res => {
      // massage response or other data processing
}).then(res => {
// fire other calls or extra processing.
}).catch((error) => {
 // handle errors.
});