How to loop API call and collect all paginated results into 1 JSON file

After the first 1000 results are loaded, I save those 1000 results in a JSON file. There is a “next_page” link.
image

When I click it, it runs the same query but for the next 1000 results (e.g. offset=1000).
image

And so on until there are no more results and therefore no “next_page” returned:
image

I then take all of these JSON files and append in powerbi, but it is a manual process to load them all into PowerBI into a table format.

Hope this helps. Ideally this would just be a solution that:

  1. Runs first query
  2. Saves JSON file
  3. Runs the “Next_page” query
  4. Append this JSON file to previous
  5. Loop until no further “next_page” query returned

Tanks!!!

1 Like