Arg.executions in `test` event

🙋 Help
1

Hi guys,

could you please explain what args.executions is referring too on the test event callback?
I printed it and tried to find the documentation of its meaning specifically for that event I can’t figure it out.

}).on('test', function (error, args) {
    console.log(args.executions.length);
}

is always returning 3 elements. What are these 3 elements?
I need to understand it because on this thread @udit suggested to use args.executions[0] to retrieve the environment variables. I just want to understand why args.executions[0] and not args.executions[1] or args.executions[2] for instance?

Below an example:

[
  {
    event: PostmanEvent { listen: 'test', script: [PostmanScript] },
    script: PostmanScript {
      id: '0b5f0147-b5ac-4489-8751-bafc6aac36f4',
      type: 'text/javascript',
      exec: [Array],
      _lastExecutionId: '40c6e1b9-478c-4a97-95ed-e92fcb64a20a'
    },
    result: {
      id: '40c6e1b9-478c-4a97-95ed-e92fcb64a20a',
      target: 'test',
      legacy: [Object],
      cursor: [Object],
      data: {},
      cookies: [],
      _variables: [PostmanVariableScope],
      environment: [PostmanVariableScope],
      collectionVariables: [PostmanVariableScope],
      globals: [PostmanVariableScope],
      request: [PostmanRequest],
      return: {},
      response: [PostmanResponse]
    }
  },
  {
    event: PostmanEvent { listen: 'test', script: [PostmanScript] },
    script: PostmanScript {
      id: '4325ec73-bdf0-4ada-af76-d86a003f6d61',
      type: 'text/javascript',
      exec: [Array],
      _lastExecutionId: '332c1c2c-913a-442e-9e0f-f893fb548087'
    },
    result: {
      id: '332c1c2c-913a-442e-9e0f-f893fb548087',
      target: 'test',
      legacy: [Object],
      cursor: [Object],
      data: {},
      cookies: [],
      _variables: [PostmanVariableScope],
      environment: [PostmanVariableScope],
      collectionVariables: [PostmanVariableScope],
      globals: [PostmanVariableScope],
      request: [PostmanRequest],
      return: {},
      response: [PostmanResponse]
    }
  },
  {
    event: PostmanEvent { listen: 'test', script: [PostmanScript] },
    script: PostmanScript {
      id: '9db33696-7fe1-4ca3-8010-dd927ebb2fbe',
      type: 'text/javascript',
      exec: [Array],
      _lastExecutionId: 'c1bf9766-b91d-4898-825b-0d38661f4a54'
    },
    result: {
      id: 'c1bf9766-b91d-4898-825b-0d38661f4a54',
      target: 'test',
      legacy: [Object],
      cursor: [Object],
      data: {},
      cookies: [],
      _variables: [PostmanVariableScope],
      environment: [PostmanVariableScope],
      collectionVariables: [PostmanVariableScope],
      globals: [PostmanVariableScope],
      request: [PostmanRequest],
      return: [Object],
      response: [PostmanResponse]
    }
  }
]

Thanks in advance
Aurélien

2

Hi @aurelien.lair,

I believe these are the results of tests scripts executed in the collection run. If you’re getting 3, I assume you have 3 test scripts.

The environment variables provided to the collection run via newman should be the same across all these test executions, so whether you pull them from the first, the last, or somewhere in-between shouldn’t make a difference.

Best,

Kevin

3

Hi Kevin and thanks for answering. About this statement

I believe these are the results of tests scripts executed in the collection run. If you’re getting 3, I assume you have 3 test scripts.

Could you precise what you meant please? Are you talking about 3 test scripts within

  • the entire collection
  • the collection on the specific folder
  • the collection on the specific folder on a specific request

Cause that log is the output after running 1 request on that folder of that collection but I do not have 3 test scripts but only 1.

pm.test("Check http status code", function () {
  pm.response.to.have.status(200);
});

I can anyway follow your suggestion about the environment variables but was curious to understand what you meant about the meaning (specifically) of that array in that context.

Thanks
Aurélien

4

@aurelien.lair,

Sure, I can explain. The collection gets stored as a JSON file. In that JSON structure, we can find script declarations at each level of the collection hierarchy (root, folder, request). Your newman options will determine what’s actually executed (e.g., entire collection, single folder, etc.).

You’re seeing 3 executions for one of a few reasons:

  1. You have 3 different requests with some value saved as a test script.
  2. You have a value saved as a test script in at the folder- or collection-level.
  3. Some combination of 1 & 2.

Now, an interesting anomaly that occurs here is that if you’ve ever entered the Tests tab of the collection, folder, or request level and saved in the app, the value of the test script in the JSON file might be an empty string (""). I believe this still shows up as an execution in this case, even though nothing was technically “executed”.

You can actually check if this anomaly is being hit by changing your log statement to:

}).on('test', (err, args) => {
  console.log(args.executions.map((ex) => ex.script.exec));
});

If you have empty test values, you’ll get an output like this:

[
  [ '' ],
  [
    'pm.test("Status code is 200", function () {',
    '    pm.response.to.have.status(200);',
    '});'
  ]
]

Notice the first element of the output array is an array with an empty string.

Hope this helps.

Best,

Kevin

5

Hi Kevin,

thanks for answering :slightly_smiling_face:
Regarding

You have a value saved as a test script in at the folder- or collection-level.

I guess you meant a use case where I am in a test script context saving a value isn’t it?
If yes then I guess it’s like you were suspecting:

pm.test("Check http status code", function () {
  pm.response.to.have.status(200);
});

var body = JSON.parse(responseBody);
postman.setEnvironmentVariable("token", body.access_token);

And indeed when I run with the console.log I get what you were saying:

[
  [ '' ],
  [ '' ],
  [
    'pm.test("Check http status code", function () {',
    '  pm.response.to.have.status(200);',
    '});',
    '',
    'var body = JSON.parse(responseBody);',
    'postman.setEnvironmentVariable("token", body.access_token);'
  ]
]

I just don’t understand well you call it “anomaly”… Is it a bad thing to do it?

Best
Aurélien

6

Hi @aurelien.lair,

I’m calling it an “anomaly”, because it’s not intuitive that an empty script would show up in the results. Arguably, it may not be a “bug”, but it is unexpected at the very least. More than likely, it’s harmless for your use case.

Best,

Kevin