# Pagination script for looping through all response pages

**URL:** <https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512>\
**Category:** Help Hub\
**Tags:** tests\
**Created:** [October 28, 2024, 2:35pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512 "2024-10-28T14:35:03Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![swl-swaldie](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@swl-swaldie](https://community.postman.com/u/swl-swaldie)\
**Post date:** [October 28, 2024, 2:35pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/1 "2024-10-28T14:35:03Z")

</div>

New to postman, and chatgpt appears to have me on the right track, but I’m not quite there.

I’m unable to fetch all available items due to the resource’s per-page limit.

The script suggested for the pre-request is as follows:

```auto
// Initialize the environment variables only if they are not set
if (!pm.environment.get("url")) {
    // Set the initial API endpoint
    pm.environment.set("url", pm.request.url.toString());
    // Set an empty list to store all page views
    pm.environment.set("all_page_views", JSON.stringify([]));
}

```

…with the post-response as follows:

```auto
// Parse the response
let responseData = pm.response.json();

// Retrieve all page views from the environment variable or initialize if not present
let allPageViews = pm.environment.get("all_page_views") 
    ? JSON.parse(pm.environment.get("all_page_views")) 
    : [];

// Append the current page's data to the list
allPageViews = allPageViews.concat(responseData);

```

```auto

// Store the updated page views in the environment variable
pm.environment.set("all_page_views", JSON.stringify(allPageViews));

// Check if there's a next page link in the headers
let linkHeader = pm.response.headers.get("Link");
if (linkHeader && linkHeader.includes('rel="next"')) {
    // Extract the URL for the next page
    let nextUrl = linkHeader.match(/<([^>]+)>;\s*rel="next"/);

console.log(nextUrl[1]);
    if (nextUrl) {
        pm.environment.set("url", nextUrl[1]);
        // Trigger the next request using the updated method
        pm.execution.setNextRequest(pm.info.requestName);
    }
} else {
    // If there's no next page, print the total number of page views retrieved
    console.log("All page views retrieved:", JSON.parse(pm.environment.get("all_page_views")));
    
    // Clear the environment variable for the URL (pagination ends)
    pm.environment.unset("url");
}

```

Adding that console.log line after setting the nextUrl variable displays what I’m guessing would be the correct next link:

 ![image](https://us1.discourse-cdn.com/getpostman/original/3X/1/5/15eb69324714f3e24f37b7db72a54d94f6f2e660.png)

However, hitting send only gets me that first page; i.e., there is no looping through subsequent iterations.

Any suggestions would be appreciated.

---

<div class="post-metadata">

**Author:** ![danny-dainton](https://sea2.discourse-cdn.com/getpostman/user_avatar/community.postman.com/danny-dainton/32/29401_2.png) [@danny-dainton](https://community.postman.com/u/danny-dainton)\
**Post date:** [October 28, 2024, 2:50pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/2 "2024-10-28T14:50:45Z")

</div>

Hey @swl-swaldie 👋🏻

Welcome to the Postman Community :postman:

As that script is using the _setNextRequest_ function to loop through the variables, you’re going to need to use the Collection Runner.

The function doesn’t work when only sending a single request.

---

<div class="post-metadata">

**Author:** ![swl-swaldie](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@swl-swaldie](https://community.postman.com/u/swl-swaldie)\
**Post date:** [October 28, 2024, 2:59pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/3 "2024-10-28T14:59:32Z")

</div>

Thank you for the quick response, @danny-dainton .

Since I’m not familiar with using the collection runner, are there any adjustments that can be made to my setup that would enable me to accomplish this with a “single” request?

---

<div class="post-metadata">

**Author:** ![danny-dainton](https://sea2.discourse-cdn.com/getpostman/user_avatar/community.postman.com/danny-dainton/32/29401_2.png) [@danny-dainton](https://community.postman.com/u/danny-dainton)\
**Post date:** [October 28, 2024, 3:06pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/4 "2024-10-28T15:06:41Z")

</div>

I’m not in a position to try that code out and I don’t know what the API is you’re using or what it’s returning.

Using the Collection Runner is quite simple and shouldn’t require any adjustments to your script.

> **[Test your API using the Collection Runner | Postman Learning Center](https://learning.postman.com/docs/collections/running-collections/intro-to-collection-runs/)**
>
> Postman is a collaboration platform for API development. Postman's features simplify each step of building an API and streamline collaboration so you can create better APIs—faster.

To use it in a single request, you’d need to adjust your script to use `pm.sendRequest` instead but it would mean a bigger refactor, not huge but as I mentioned, I can’t really help you out with that right now.

---

<div class="post-metadata">

**Author:** ![swl-swaldie](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@swl-swaldie](https://community.postman.com/u/swl-swaldie)\
**Post date:** [October 28, 2024, 5:36pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/5 "2024-10-28T17:36:17Z")

</div>

Thanks again, @danny-dainton !  
I’ll look into that.

---

<div class="post-metadata">

**Author:** ![swl-swaldie](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@swl-swaldie](https://community.postman.com/u/swl-swaldie)\
**Post date:** [October 28, 2024, 8:38pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/6 "2024-10-28T20:38:30Z")

</div>

I read through this documentation, and it sounds like the purpose is to

> Send some or all of the API requests in a Postman Collection in the order…

However, I only have this one request, but it needs to be resent for each of the _next_ URLs that are returned in the link header until all of the pages have been exhausted.

Can either the pre-request or post-response scripts be set up to resend this request?

---

<div class="post-metadata">

**Author:** ![danny-dainton](https://sea2.discourse-cdn.com/getpostman/user_avatar/community.postman.com/danny-dainton/32/29401_2.png) [@danny-dainton](https://community.postman.com/u/danny-dainton)\
**Post date:** [October 28, 2024, 8:53pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/7 "2024-10-28T20:53:53Z")

</div>

Did you try to run your Collection with the Collection Runner?

Did it not do what you expected it too?

---

<div class="post-metadata">

**Author:** ![swl-swaldie](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@swl-swaldie](https://community.postman.com/u/swl-swaldie)\
**Post date:** [October 29, 2024, 1:55pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/8 "2024-10-29T13:55:03Z")

</div>

I did, but it just keeps running the same request; i.e., the “next” url is not being used, and it never stops.  
I’m going to add a few more log statements to see where i’m going wrong.

---

<div class="post-metadata">

**Author:** ![gbadebo-bello](https://sea2.discourse-cdn.com/getpostman/user_avatar/community.postman.com/gbadebo-bello/32/12175_2.png) [@gbadebo-bello](https://community.postman.com/u/gbadebo-bello)\
**Post date:** [October 30, 2024, 5:52pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/9 "2024-10-30T17:52:15Z")

</div>

Hi @swl-swaldie. Can you share a screenshot of what your script currently looks like?

---

<div class="post-metadata">

**Author:** ![swl-swaldie](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@swl-swaldie](https://community.postman.com/u/swl-swaldie)\
**Post date:** [October 30, 2024, 6:01pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/10 "2024-10-30T18:01:39Z")

</div>

Hi Gbadebo,  
Here’s the pre:

 ![image](https://us1.discourse-cdn.com/getpostman/original/3X/a/e/ae88f29bbc2b999c9fb52eadc209308e951f2d38.png)

And here’s the post:

 ![image](https://us1.discourse-cdn.com/getpostman/original/3X/2/f/2fd314549f72b8a3ace2db4e9e19b893be2606d0.png)

I’m currently trying to figure out a way to get the request to GET the value of  
`nextUrl[1]` after the first iteration, and keep looping until there are no more pages to fetch.

---

<div class="post-metadata">

**Author:** ![swl-swaldie](https://avatars.discourse-cdn.com/v4/letter/s/a587f6/32.png) [@swl-swaldie](https://community.postman.com/u/swl-swaldie)\
**Post date:** [November 6, 2024, 4:11pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/11 "2024-11-06T16:11:44Z")

</div>

After exhausting my efforts with chatgpt, I switched to copilot and I finally arrived at a solution using the following scripts:

**Pre-request**

```auto
if (!pm.collectionVariables.get("allResponses")) {
    pm.collectionVariables.set("allResponses", JSON.stringify([]));
}

```

**Post-response**

```const
let iterationCount = 0;

function getNextPageUrl(linkHeader) {
    const links = linkHeader.split(',');
    let nextPageUrl = null;

    links.forEach(link => {
        const [url, rel] = link.split(';');
        if (rel.includes('rel="next"')) {
            nextPageUrl = url.trim().slice(1, -1); // Remove angle brackets
        }
    });

    return nextPageUrl;
}

function processResponse(res, url) {
    if (!res || !res.headers) {
        console.error("Invalid response object:", res);
        return;
    }

    iterationCount++;
    let allResponses = JSON.parse(pm.collectionVariables.get("allResponses"));
    try {
        const responseBody = res.json();
        
        // Assuming responseBody is already an array of objects
        allResponses = allResponses.concat(responseBody);
        pm.collectionVariables.set("allResponses", JSON.stringify(allResponses));
    } catch (e) {
        console.error("Failed to parse JSON or extract data:", e);
    }

    const linkHeader = res.headers.get('link');

    if (linkHeader) {
        const nextPageUrl = getNextPageUrl(linkHeader);

        if (nextPageUrl && iterationCount < maxIterations) {
            const authHeader = pm.request.headers.find(h => h.key === 'Authorization');

            pm.sendRequest({
                url: nextPageUrl,
                method: pm.request.method,
                header: {
                    'Authorization': authHeader ? authHeader.value : '',
                    ...pm.request.headers
                },
                body: pm.request.body
            }, function (err, newRes) {
                if (err) {
                    console.error("Error on next request:", err);
                } else {
                    processResponse(newRes, nextPageUrl);
                }
            });
        } else {
            console.log("All pages fetched. You can now export the data from the Postman collection variable.");
        }
    } else {
        console.log("All pages fetched. You can now export the data from the Postman collection variable.");
    }
}

// Initial call to process the first response
processResponse(pm.response, pm.request.url.toString());

```

The full response body is then captured in the current value of the _allResponses_ collection variable:

 ![image](https://us1.discourse-cdn.com/getpostman/original/3X/c/7/c7985780ac1c48fa7644df93c0afa6a1864c94dd.png)

And this did not require use of the runner functionality.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/getpostman/user_avatar/community.postman.com/system/32/29796_2.png) [@system](https://community.postman.com/u/system)\
**Post date:** [December 6, 2024, 4:11pm UTC](https://community.postman.com/t/pagination-script-for-looping-through-all-response-pages/70512/12 "2024-12-06T16:11:51Z")

</div>

Closed on December 6, 2024, 4:11pm UTC

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.