Is there a way to see all collection run results from all sources for a source controlled API collection?
There is no option to specify the branch or version when running the tests via the Postman CLI. Does it just use the “latest” published version? If so how are these results accessible afterwards?
Hey @tristanmisselbrook!
So when connecting a Git-based version control repository and importing your collection, you have to select which branch to use. Once connected you can switch branches and perform all of the other typical tasks such as pulling, pushing etc.
When it comes to the Postman CLI and Newman, it uses the collection file that you specify. So it doesn’t really care about the branches or the versions, it just simply uses the file that you provide. You would have to checkout to what ever other branch first in order to change what is being run.
As for your other Q, I’m not sure I’m following, but if you leave the “Source” filter un-defined, it would show results for all sources connected to your collection. If this doesn’t fit your use case, you can also store these results externally after running via Newman.
Thanks for your reply @kevinc-postman
I run the collection via postman collection run <collection-id> which I assume pulls the collection data from postman but I have no indication of what branch this might be as the id is the same for each one.
Then there are no runs listed in the GUI for the collection under any branch. Additionally there is no “Runs” tab under the published version so I cannot see if they are appearing there. Is there anywhere that I can view these?
I notice that the results from the link in the CLI output are actually listed under a develop branch that doesn’t even exist for the repository (screenshot below). Is this some sort of postman default?
but I have no indication of what branch this might be as the id is the same for each one.
This should run the currently checked out version / branch of your collection that is reflected in Postman.
As for the develop branch, could you try removing the Git link and reconnecting to the repo to update your branches?
This should run the currently checked out version / branch of your collection that is reflected in Postman.
Even running through the CLI?
I did some testing and it uses the latest published version by default and reports them against the develop branch. This seems like weird behaviour, possibly a bug or missing feature?
We will just add a develop branch as a work around, and will have to keep publishing versions every time we make a change to our test collections. Although that is less than ideal as it leaves us with no way to do on branch testing through the CLI, without updating everyone’s version. Using the built-in collection runner is not an option after the limit changes.
You’re correct that the Postman CLI doesn’t natively support fetching different versions or branches of a collection. This is because collections in Postman are not maintained like a codebase in a Git repository. In Postman, when you save changes to a collection, those changes become the ‘current’ version of that collection, regardless of branch. It’s more of a single active ‘state’ system than a repository of multiple accessible versions or branches.
When you run a collection with Newman or the Postman CLI, it grabs the current saved state from Postman, which may be the latest published version. But it doesn’t inherently know or care about branches; it’s working with the current active state of your collection.
So when you run a collection using the CLI, it uses the state of the collection as it is currently saved in Postman. If you have multiple branches in your Postman collection, you have to manually switch to the branch you want to run, save the collection, and then run it.
Now, for your develop branch workaround, I get why you’re thinking of doing that. But there might be a simpler way. What if you exported the specific version of the collection you want to test and then fed that file into Newman?
That way, you can run tests on any branch, at any time, without having to disturb the current active state of your collection in Postman. And no need to add or manage extra branches just for testing!
This method, however, means you need to manually export the collection each time you switch branches, which could be a pain depending on your workflow. I totally get it if this isn’t a perfect solution, but it might help in the interim.
As for seeing all collection run results from all sources, you could integrate Postman with a CI/CD tool like Jenkins or Travis CI. Then, you can store and manage your test results more effectively. Also, Newman and the Postman CLI offers several ways to output test results, such as JSON, XML, HTML, etc. This can give you more control over how you store and view your results.
