Controlling variable scope to request level

[morleym_cubic]

Describe the issue
Team members are using environment variables to set request variable values in their pre-request scripts (not just headers, URL endpoints, etc.). This, as expected, is causing scope pollution with bloated env files, headache collisions to debug, etc.

For example, to generate a unique email at the time of sending a request, team members are:

1. Generating a unique email value in the pre-request script
2. Assigning this value to an environment variable in the pre-request script
3. Using the environment variable value in the key-value pairs in the request body

The best practice is to scope variables to as specific a level as possible. Despite digging through documentation and searching forums, I have found no way to access and update any variables used in the request body from the pre-request script.

There are no repro steps or expected behavior to list because it is a matter of non-existent (or not yet found?) features.

What should it look like?

What I want is to have something like this in a pre-request script:

const randomlyGeneratedEmail = () => //returns String of email using base and timestamp;
const email = randomlyGeneratedEmail();
pm.setRequestVariable('email', email); //this method does not exist

And then in the body of the request I would have this key-value pair of “email”: “{{email}}” which would use the value I set in the pre-request script.

Either that, or I should be able to access the pre-request script variables from the key-value pairs in the request body. What we’re doing now does not work and I need a working solution for variable scope.

[singhsivcan (Sivcan Singh)]

Hello @morleym_cubic, welcome to the community!

There are two things that can help you out.

1. Sessions

You can make your team-members TURN OFF the “Automatically Persist Variables” in the Settings and then it will no longer pollute the environment until a team member actually decides to persist the values.

2. Local Variables in a request

Instead of using the environment variables, you can use local variables.
These variables are only available during the execution of the request and aren’t persisted anywhere.

For eg:

// Pre-request script
const randomlyGeneratedEmail = () => return 'someRandomEmail';
const email = randomlyGeneratedEmail();
pm.variables.set('email', email);

Now, you can use this variable in your request anywhere as you would use a variable with curly braces.

For eg.:

https://postman-echo.com/get?foo={{email}}

And it’ll be resolved during execution and will not be persisted anywhere.

[morleym_cubic]

This is GREAT for the single-request use case, thank you!

In the event that I need to set a variable in a pre-request script or a post-request test script, and use it in the next request in the collection, is there a way I can do this without using environment variables?

For example, let’s say I get a unique (for every time I run it) account ID response from my first request in the collection, and then I need to use that account ID in the second request in the collection. Can I do this without using environment variables?

[singhsivcan (Sivcan Singh)]

@morleym_cubic There are only two things, that are environment variables and global variables, and only these can help you with your use-case.

But I am pretty sure that using sessions would help you out along with the settings to automatically persist variables turned off for everyone so that the main environment is never polluted and each developer can work on their own session and update variables accordingly.

---

Another thing:
If you use the collection runner, then you can run the complete collection using the environment and make sure the “Keep variable values” option is turned off.

What this is gonna do is it’ll use the environment variables in a sandboxed way and all your requests will function as you’d want them, but after the collection run finishes - it will not update the existing environment that you actually have in the app.
So, your variables will never update in the main environment but yeah during the collection run, they’ll definitely get updated in the sandboxed env.

[morleym_cubic]

There are only two things, that are environment variables and global variables , and only these can help you with your use-case.

@singhsivcan that is actually incorrect! You CAN use/manipulate collection-level variables across requests in that collection without creating/using environment variables. I tested out a solution yesterday where I declared collection-level variables (in Postman GUI, go to collection → edit → Variables tab and define the variables there) and set/get them using pm.variables in the pre-request/test scripts of your requests in that collection.

*While this crossed-out bit is functionally accurate today (using/manipulating a variable throughout a collection), it is misleading (it was a ‘local’ variable being used across the execution of the collection, and NOT an actual ‘collection variable’ being set). If you have the same problem as me, the information in this solution WILL solve your problem today but I would expect that as time goes on, there will be a better/cleaner solution.

As multiple collections may use the same env variables, being strict on scope pollution drove me to this solution. I think leaning on env variables is not good practice. Instead, one should always scope as specifically as possible.

Even if you can turn off a setting to persist, you’re still bloating an env file with non-static data which is created at the time of running your tests. For readability and source control purposes, env files should be limited to static data only (service endpoints and ports, the required password length for accounts in that env, etc.).

To Use a Variable Throughout a Collection (and use in requests throughout that collection)

1. First, define the variable in the collection. Hover over your collection, and click the three dots (‘view more actions’). From there, click the ‘Variables’ tab and define your collection variable(s) here. *This step is actually 100% extraneous/unnecessary, but I think this is the more forward-looking/future-proof way to proceed today (assuming that Postman eventually has a method to manipulate collection variables)

2. Use *the variable(s) in the pre-request/test scripts for requests in that collection using pm.variables.get('variableName', value) or pm.variables.set('variableName', value), or reference it in request contents (url, headers, body, etc.) with {{variableName}}

3. Open the postman console, and run your collection (not each individual request one-at-a-time, but actually run the full collection using the collection runner). Trace the value of your variable(s) across the requests in that run of the collection to confirm that it’s working as you expect. Unfortunately, at this time there won’t be a way for you to go one request at a time manually and use a variable through the collection that way.

Thanks again @singhsivcan for the help!

[singhsivcan (Sivcan Singh)]

I am very well aware of collection variables but since you wanted to modify the variables using the scripts, that isn’t possible using the collection variables because they’re READ-ONLY during the request execution phase.

As you mentioned above:

In the event that I need to set a variable in a pre-request script or a post-request test script, and use it in the next request in the collection, is there a way I can do this without using environment variables?

pm.variables API actually resolves using scope so all scopes of the variables are resolved.
And when you’re writing pm.variables.set you’re actually setting a local variable and since LOCAL VARIABLE has higher precedence over the other variables, this is why it’ll be showing up as resolved. So in the end, you are not modifying the collection variable.

Variable resolution order:

1. Global (Lowest)
2. Collection (Read-only)
3. Environment
4. Data
5. Local (Highest)

So when you hit send / write pm.variables.get in the scripts, the variables are resolved in the above specified order.

So, to sum it up - you can only use read-only collection variables for the entire collection and to modify them, you need to Edit the collection and modify them.

Again, for your exact use-case which you explained was to modify variables using the scripts and using them across the requests, you should use environment and global variables, and when you’re writing pm.variables.set you’re actually setting the local variable which is local to the request.

For eg:
Define a collection variable counter and set its value to 1.
Now, in pre-request script of first request write:

pm.variables.set('counter', 2); // This becomes a local variable and doesn't modify the collection variable.

Now, take another request in the collection and read the value of counter using pm.variable.get('counter'), you’ll actually get 1 again.

Here’s a collection which you can import and test it out:
https://www.getpostman.com/collections/ff9de1b8ca142832ee42

Run the first request, see the value of counter.
Then run the second request and you can check the value of the counter.

You can read more about variables and the scopes here: https://learning.getpostman.com/docs/postman/environments_and_globals/variables/

[morleym_cubic]

Now, take another request in the collection and read the value of counter using pm.variable.get('counter') , you’ll actually get 1 again.

@singhsivcan This is why it’s important that you read my note about running the full collection.

Look at what happens with your example when you do that! (for anyone who reads this topic later, the request named “Set variable to 1” is actually setting the variable to 2 in its pre-request script so it’s not named very aptly)

[morleym_cubic]

@singhsivcan Also to be fair, it seems none of this is documented so I don’t blame you for not knowing this interaction is possible! I had to do this with desperate exploratory testing and I’m just happy it works. Hopefully there will be documentation soon for it.

[singhsivcan (Sivcan Singh)]

Well, it was an example but thank you for pointing out the mistake, I have updated the collection name.

Let me explain it in a deeper context:

1. The local variables stay during the execution of the request in a sandbox (ran by the postman-runtime), and since the collection runner will run all the requests in the sandbox, the variables exist.
   But the lifetime of the variable is dependent on where it executes and the sandboxed environment.
   Yeah, in this case, the scope of the LOCAL VARIABLES exist, but they are not COLLECTION VARIABLES.

Actually, local scope is much more complex than what’s mentioned so far. It works similar to prototype in JavaScript. Example:

// pm.variables.get('foo') will be looked up like this:
{
  state: [state from previous runs], // return `foo` from local scope
  // otherwise, lookup all the layers until `foo` is found
  layers: [
   "VariableScope:data",  // return `foo` from data scope else keep searching
   "VariableScope:environment", // return `foo` from data scope else keep searching
   "VariableScope:collection",  // return `foo` from collection scope else keep searching
   "VariableScope:global",  // return `foo` from global scope else undefined
  ]
} 

Now this local state it holds is persisted across the collection execution.

Already pointed that out and the team will come up with a more detailed explanation around this in the postman-sandbox repository in the future.

It’s great that using the local variables, you’ve solved your use-case in the collection runner but the whole context that I am trying to convey here is that you’re not modifying the collection variable using the script.

2. When you would run this through the request builder, you can see it here:
   Collection Variables.mov - Google Drive

I understand the documentation doesn’t quite explain all these depths and there’s definitely scope for improvement there, but for now again - You cannot manipulate collection variables using the scripts.

It’ll be good if we close this thread so that it doesn’t confuse anyone and also, I’ll give feedback to my team regarding the documentation and we’ll look into improving it and explaining these cases too.

There’s definitely a gap here and maybe in the future, there might be support for finally modifying the collection variables using the scripts too.

Thank you for your time and your contribution to the community!

[morleym_cubic]

@singhsivcan I apologize, I understand what you’re saying now.

This is very strange- so local isn’t just request-scope, but it’s actually *effectively collection-scope for the purpose of running a full collection. I definitely misunderstood that from the current documentation.

I hope that things get cleared up in documentation/functionality later! My ideal workflow would be what I’m doing, but having a method to explicitly set those collection variables. My reasoning is that if I don’t currently define it in the collection, then the references don’t look like they’re going to work in the later requests.

I agree this thread is probably going to be confusing to folks because of my misunderstanding but hopefully will at least unblock someone who ends up with the same needs/problems as me.

[singhsivcan (Sivcan Singh)]

Awesome, I am glad it cleared the confusion.

I am sure a lot of people will stumble upon this and will be able to understand the complexities around this.

Will be closing this thread now!