After the first 1000 results are loaded, I save those 1000 results in a JSON file. There is a “next_page” link.
When I click it, it runs the same query but for the next 1000 results (e.g. offset=1000).
And so on until there are no more results and therefore no “next_page” returned:
I then take all of these JSON files and append in powerbi, but it is a manual process to load them all into PowerBI into a table format.
Hope this helps. Ideally this would just be a solution that:
- Runs first query
- Saves JSON file
- Runs the “Next_page” query
- Append this JSON file to previous
- Loop until no further “next_page” query returned
Tanks!!!