With Version 11 and the introduction of the Package Libraries we are very exicited to start using the Package Libraries.
Currently we run >7000 Requests (~40 Collections, located in 4 different Workspaces in Postman), through Azure pipeline for our Automation in Lab and Staging Envioronments.
We are currently using Newman and a junit reporter which outputs the run results in json which is then convered to XML which we then use to populate a the results in Azure.
Unfortunatly Newman doesn’t support Package Libraries and it seems that Postman CLI doens’t support the "-r) switch to run reporters.
So we are in a fix…
Trying to glean test results out of the Postman UI is not a viable solution.
If anyone has a solution to getting the results of a Postman CLI Run through an Azure Pipeline It would be Greatly Appreciated
We currently use this plugin to run newman
and this is the reporter we use:
If we could get this reporter to run for Postman CLI, we would be in Great Shape.
Postman CLI currently does not support plugins or exporting of run results like newman does. This is something we’re well aware of and are looking into providing soon.
I’m unable to share an ETA at the moment, but we’re looking into this internally and I’ll do well to update this thread when I’m able to communicate an ETA for it being available.
Thank you so much for the information!
I wonder if there is a possibility to have extra reports in the Azure pipelines?
How the task in the pipeline yaml should look like to have the “nicer” report on the Azure?
Could you expand a bit more on what you mean here? Are you referring to wanting the report in a specific format? If yes, what format works best for Azure pipelines?
I remember that some years ago when I used Newman to embed Postman tests in the pipelines in Azure Devops there was a possibility to add steps that would publish test results in a user-friendly format like these 2 examples. Does Postman CLI permits something similar and what would I need to write in yaml to obtain such results?
Can you explain what your pipeline looks like? It should now be similar to how you would set it up in Newman.
I notice you have Test Plans enabled within your ADO project.
You should be able to publish the test results directly to Test Plans.
Start by getting your sources. With the CLI, you might not need this step because you can use command line options to pull the collection straight from Postman Cloud.
Then your pipeline would have three main steps:
Install the CLI tool.
Run your Postman collection. Make sure the working directory is set correctly, because you’ll need it later when publishing the test results.
Publish the test results using the “Publish Test Results” action. This will upload the results to the Test Plans section in Azure DevOps (ADO).
Just make sure the format of the test results matches the format expected by the reporter. I would try the JUnit option.
One important detail: the working directory must be set properly so the report gets picked up correctly.
Thank you for your reply! Well, our pipelines are defined not in Azure but in our monolith codebase in the .yml files. Therefore, I have difficulties using a task of the classic editor in Azure pipelines (the screenshot you provided). I would need to know the lines in yaml that would do the publishing…
.
