Chaining two requests with a token passed from the first to the second query

magnus_r · December 16, 2020, 3:21pm

Hello

New to Postman but enjoying the learning journey a lot.

I have two queries where the first one “BeginQuery” returns a token that is next used in “EndQuery”. I have not been able to have EndQuery wait for the token (which set in BeginQuery’s test step ) until EndQuery is sent. Without a token the EndQuery fails.

If I hardcode a five second timeout in EndQuery¨s pre-request script I can run the the collection just fine. But I would much rather have a condition that waits for a collection variable to be set before running EndQuery. I will build many other tests for other endpoints what may wait longer or shorter. It also make sense to be able to set a max timeout if possible.

I have tried some if else statements with timeouts and a while loop (that went infinite loop on me). but so far no success. I have tried to illustrate the steps below.

BeginQuery

Pre-request script:
none

Query:
send some known data

Test:
run various tests
set a collection variable for the token returned from the response pm.collectionVariables.set(“token”, $(‘BeginStrategyResult > Token’).text());`

(I have checked that the collectionvariable "token"is in fact created at this step and later unset after EndQuery test step)

EndQuery

Pre-request script:
Wait until the collection variable “token” is set. This is where I run into the problem
pm.collectionVariables.has("token")

Query:
Send token in request

Test:
run various tests
clear the token variable
pm.collectionVariables.unset("token");

I appreciate any help the community can offer. Cheers!

danny-dainton · December 16, 2020, 5:59pm

Hey @magnus_r

Happy to help you along this journey!

Could you provide an example response body for the first request that captures the token, please?

The second argument here doesn’t look quite right to me but wanted to confirm that it’s setting the value correctly.

pm.collectionVariables.set(“token”, $(‘BeginStrategyResult > Token’).text());`

In your second pre-request script, you could log out the value to ensure this is being set.

console.log(pm.collectionVariables.get("token"))

magnus_r · December 17, 2020, 7:05am

Hi @danny-dainton

The reason the arguments look strange is that this is a SOAP request. I should have mentioned that from the start of course. Below is part of the response

      <BeginStrategyResult xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
	    <Token>3c5cf28a-436b-48cd-b75e-18ce81144140</Token>
	 </BeginStrategyResult>

I capture the token using the cheerio library like so in BeginQuery’s test step.

const $ = cheerio.load(responseBody);
pm.collectionVariables.set("token", $('BeginStrategyResult > Token').text());

And the token variable is set:

I have a number of tests well as the code that sets the token variable. It takes about 5 seconds to set .

The EndQuery will need the token in its request body.

             <Token>{{token}}<Token>

And in the EndQuery’s test phase the token variable is finally unset,

// Clear the token from BeginQuery at every test run
    pm.collectionVariables.unset("token");

With every new BeginQuery the token variable is set anew.

If I manually run BeginQuery, wait until its test scripts are run and the new collection variable is set, and then send EndQuery I can confirm that everything works from beginning to end.

In EndQuery’s pre-request scripts I can hardcode the wait like so:

setTimeout(function() {}, 5000); // Hardcoded wait 5 sec

My question is how I automate that wait in a smart way. Yesterday I tried creating a while loop that will loop as long as the token variable is not set… The loop keeps checking every second. Once the variable is set the pre-request scrip completes and the EndQuery is sent. That was the intent anyway. I don’t often use JavaScript and it probably shows.

i = 0
while(!pm.collectionVariables.has("token")) {
    console.log("No token yet");
    i ++;
    setTimeout(function(){}, 1000);
    if(i >=10) {
        console.log("Taking too long. Breaking out of the loop");
        break;
    }
}
console.log("The end!");

Just to be sure: If I run EndQuery on its own is does unset the token variable:

But when I run the collection EndQuery runs too early and does not pick up the token variable. I checked console log and the only output is “The end”.

This is the collection runner

I hope that clarifies what I have tried and where I am stuck. I really appreciate you taking a look.

danny-dainton · December 17, 2020, 7:19am

So it’s the request to get the token that takes a while to return?

The token value would be set straight away, if you were to hard coded that value.

What’s the response time of the first request? I thing that’s the part of unclear about

I see that you’re using the Collection Runner, can you ensure that for each change you make to any single request, you save it before using the Runner. Without doing that, it won’t know about the changes to the request.

magnus_r · December 17, 2020, 8:01am

I am using collection runner, The two queries are in a way pointless on their own, so they should definitely be run in a pair one after the other, passing the token from beginning to end.

Whenever I try to console log the token variable before it is set I get an error and the collection runner stops. You mentioned I could hardcode the token variable. That’s probably better than setting and unsetting the variable each time. I will hardcode the token variable and change both tests and pre-request scripts accordingly.

danny-dainton · December 17, 2020, 8:11am

That’s not really what I said or meant

I understand what you’re trying to do - How long does the token request take to respond and return the value? That’s the bit that’s confusing at the moment as that test script should run after that’s returned a response.

Not sure why you would need a delay or think that’s something isn’t set in time on that request, unless there is something else going on that you’re not saying

Hardcoding something was just an example, that would show that you a value is set as a variable straight away, there shouldn’t be any kind of delay in the way it sets the variable.

magnus_r · December 17, 2020, 8:30am

the BeginQuery takes about 200 milliseconds

But the thing is, there is not yet a token variable. (collection variable to be specific) That variable is set only after the BeginQuery’s tests have run.

I need EndQuery to run, not just after BeginQuery receives a response, but only after the collection variable is set, and that takes considerably longer, about 5 seconds what I have seen.

The whole logic of passing the token from one query to the other I have learned from a great YouTube vide from @vdespa Kudos BTW!

But the video does not go into my problem of making sure the next query waits until the collection variable is ready.

danny-dainton · December 17, 2020, 8:44am

Capturing the token would be part of that same requests test script though right?

At the point of those tests running, can you see the token value you want in the response body? If its there and the way you capture the value is correct, it should set that straightaway without any delay.

Maybe there some additional context here I’m missing, I could understand a delay setting the variable being there, if you have explicitly added one but not without doing that.

It’s a tricky one to walk through as you can see all the things in front of you that I can’t so there could possibly be something else being over looked.

magnus_r · December 17, 2020, 9:02am

Yes indeed.

Below is the entire test script for BeginQuery. I believe these assertions take up the quite a bit of time. Only at the last line is the token collection variable set.

pm.**test**("Status code is 200", **function** () {
  pm.response.to.have.status(200);
});


/*
pm.test("A token is returned", function() {
  const $ = cheerio.load(responseBody);
  pm.expect($('BeginStrategyResult > Token ').text()).to.not.be.empty;
});
*/


pm.**test**("A token is returned", **function**() {
  const $ **=** cheerio.**load**(responseBody);
  pm.expect($('BeginStrategyResult > Token ').text()).to.be.length(36);
});

const $ **=** cheerio.**load**(responseBody);

pm.collectionVariables.**set**("token", $('BeginStrategyResult > Token').text());

If I try to console log an non-existing variable the collection runner stops. I was about to set a hardcoded token variable (shorter that 36 chars) but is that good practice or not? Is it preferable to set and unset the collection variable at each collection run?

danny-dainton · December 17, 2020, 9:40am

Out of interested, does that work:

const $ = cheerio.load(pm.response.text());

pm.test("Status code is 200", function () {
  pm.response.to.have.status(200);
});

pm.test("A token is returned", function() {
  pm.expect($('BeginStrategyResult > Token ').text()).to.be.length(36);
});

pm.collectionVariables.set("token", $('BeginStrategyResult > Token').text());

or this:

const response = xml2Json(pm.response.text())

pm.test("Status code is 200", function () {
  pm.response.to.have.status(200);
});

pm.test("A token is returned", function() {
  pm.expect(response.BeginStrategyResult.Token).to.be.length(36);
});

pm.collectionVariables.set("token", response.BeginStrategyResult.Token);

In the test script you posted you’re declaring the same $ variable twice and that should throw a script error.

magnus_r · December 17, 2020, 2:00pm

Everything works when it comes to setting the token, changing its value and asserting the value. I don’t want to convert xml to json either as I use the cheerio library (that uses jQuery syntaxt) to traverse the xml DOM successfully.

Now I understand that the heart of problem is that I don’t know how to code a JavaScript timeout() to run synchronously. Out of the gate JavaScript setTimeout() is asynchronous. It only delays the code in the function, not the whole script.

Below is my current attempt:

function checkToken () {
  t = pm.collectionVariables.get("token");
  if (t.length == 36 ) {
     console.log("36 !!!")
  } else {
    setTimeout(checkToken, 1000); // try again in a second
  }
}

checkToken();

magnus_r · December 17, 2020, 3:12pm

I checked with a colleague and I have probably looked at the problem in the wrong way and made incorrect assumptions on how collection runner works. I will probably have to settle for a hardcode pause between the two queries. I will update the thread with a better explanation tomorrow after I have refactored the tests.

danny-dainton · December 17, 2020, 6:08pm

That’s fine - the first script should work for you then.

I need to say this again, just in case you missed it - If you make any change in the request ensure to save all the changes before running the Collection in the runner.

If you have the Collection open in the runner and you go and make a change to any request, if those changes are not saved - The runner will not know anything about them and run the collection in the previous state.

I’m still unsure why it would be taking so long to set the variable but without seeing all the scripts that you might have in all of the collection - I’m just making guessing. You really shouldn’t need to add in functions to check if a variable is set of not, if the request has executed correctly the variable should be there straightaway.

magnus_r · December 18, 2020, 8:19am

Ok. I my colleague found a root cause of the problem after some troubleshooting.

The token is actually set and sent correctly, it is just not ready to be consumed. If I send the token immediately after receiving it I get a 500 error. and part of the error message says something like this:

    <Fault>
        <faultcode>Client</faultcode>
        <faultstring">Query is still in progress.</faultstring>
        <detail>
            <InProgress xmlns="http://redacted.com/schemas/2012/09/Redacted"
                xmlns:i="http://www.w3.org/2001/XMLSchema-instance"/>
        </detail>
    </Fault>

The solution right now is to retry the request until I get a successful response.

I added the following code to the EndQuery’s test.

if(pm.response.code !== 200) {
    setTimeout(() => {}, 1000);
    postman.setNextRequest("EndQuery");
} else {
    postman.setNextRequest(null);
    pm.collectionVariables.set("token", "hardcoded_token");
}

I will refactor this code to avoid infinite loops etc, but the basic logic works.

All in all I had a case of tunnel vision, thinking the token was not sent. I should have looked closely at what was sent and the contents of the 500 response.

Thank you very much @danny-dainton I haven’t been able to share requests and responses with you, and I also had misleading assumptions about the problem I was facing. You have been very helpful and patient all along.