How to run a monitor multiple times with different data

🙋 Help
, , ,
1

We are building a Postman solution to monitor API request performance for our organization by charting the latency over time of a collections of requests. Our application can have an arbitrary number of user generated projects. The latency of requests can vary depending on how a user configures each project. To profile this, we would like to first pull a list of available projects from our application, and run the collection of requests on each of them.

We currently accomplish this with a setNextRequest loop, which iterates through all available projects. I’ve added a few code snippets below to help explain:

// the start of the loop. we get number of projects and data from the current project: 
const projectList = JSON.parse(responseBody);
const pidx = pm.environment.get("projectIndex");

pm.environment.set("numProjects", projectList.length);
pm.environment.set("projectId", projectList[pidx].resourceId);
pm.environment.set("projectTitle", projectList[pidx].name);

// we can now use {{projectId}} in a few requests we want to test.

// and in the last request we either go back to the start of the loop or break:
pm.environment.set("projectIndex", pm.environment.get("projectIndex") + 1);
pidx = pm.environment.get("projectIndex");
pnum = pm.environment.get("numProjects");

if (pidx < pnum) {
    postman.setNextRequest("beginning");
}
else {
    postman.setNextRequest(null);
}

This issue with this solution is that there is not a good way to associate each request to the current project name. At the moment, we output the project name as a conditionless test on each request:

pm.test(`Project: ${pm.environment.get("projectTitle")}`);

However, this feels a bit hacky, and this test data does not persist through to our datadog integration. Additionally, in the monitor UI, all requests with the same request names have their latencies summed, even if each individual request tests different projects. We would like our monitor results, either in Postman or Datadog, to chart the results for each request separately if the variables used to call the request are different even though the request name is the same. If we could dynamically change the request name that would be one solution but we have yet to find a way to do this.

Is there a better way of doing what we are trying to do?

3 Likes
2

Hey @sxk003 :wave:

I could manage to run a single collection while dynamically updating request names utilizing Postman API “Update Collection” endpoint.

The tricky part here is however that if we update request names while running a monitor / collection runner, request name stored in pm.info.requestName is not updated (and this request name seems to be the one used in Monitor / Collection runner reporting). This means that we need to run a Monitor separately for each request name update.

So what I did is:

  1. Create a monitor for a collection we want to run and dynamically update request names
  2. Run the monitor via Postman API “Run a Monitor” endpoint, stored in a separate collection

Demo: Loom | Free Screen & Video Recording Software

In this example, I am updating request names such that “Request 1” → “Request 2” → “Request 3” while executing a monitor run each time.

Please refer to the following sample collections used in this demo.

This is a hacky way, and there might be better ways to do this, but hopefully this is helpful :slightly_smiling_face:

2 Likes
3

While definitely a tad hacky, this solution ended up working perfectly for us. Thanks for putting this example together! For anyone else trying the same thing, keep in mind that if you want an updated environment to persist across different monitor runs, you will have to update the environment through the postman API instead of the JS call.

4

Great to hear that it worked :raised_hands:
Also thanks for sharing your findings about environment updates with other users :slightly_smiling_face: