I’m working on a flow that needs to process a large array of JSON objects fetched from an API. The response is stored in the body variable of a GET request, containing over 1000 JSON objects. I’m encountering out of memory errors when trying to process this data. Here’s my current approach:
Send a GET request to fetch the data (resulting in one large array with 1000+ JSON objects in the response body).
Select the body of the response for further processing.
Use a For loop to iterate through all elements in the array.
Evaluate each use a TypeScript script to perform checks (e.g., verifying if the body is an array, fetching certain values, etc.).
However, I’m getting out of memory errors in the first Evaluate step.
The API doesn’t currently support pagination, though I plan to request this feature from the provider.
My question is: Is it possible to chunk this large array into smaller pieces within Postman Flows and then process these smaller chunks? I’m looking for a way to handle this data without running into memory issues.
Are you running this locally by pressing the “run” button on the flow, not via a webhook on Postman’s cloud?
Since the request executes, but the flow fails in the evaluate block inside the for loop, could you try slicing the array before passing it into the for loop to just verify a subset of the data like 10 records works successfully? (in TypeScript: array.slice(0, 10))
Thank you for your response. Unfortunately, we don’t have a workaround for this issue at the moment, as we need to load the array into memory to process it.
I spoke with our backend team, and we will increase the memory available to the evaluate block to help address this issue. However, I don’t have a specific ETA on when this will be completed.
In the meantime, we hope the API will add pagination support.
