I’ve set up a pipeline with Jenkins that uses Docker to build and compose some services I created when I make a new Github Pull Request and it automatically runs a Postman collection using Newman and reports to Github whether or not it should block the merge depending on the test results.
Jenkins uses the API key of the collection that I generated in order to fetch the collection’s latest state. This seems to work very well and the pipeline is solid.
However, the problem I ran into is when someone’s PR makes some tests fail. That happens when their PR changes how a service’s API functions. If that is unintentional, then everything is great, they just need to fix the bug in their code. However, if it is indeed intentional and they need to update the Postman tests we have an issue:
- If they change the Postman tests to reflect how the API now should work, if we run the collection on the Master branch the tests they changed will now fail, even though for the master branch release everything should still be fine (they haven’t merged the PR yet)
The solution I thought of was to utilise forks. Namely, if the Newman check in the developer’s PR fails and they realise they need to update the Postman tests, they could fork the Postman collection, fix the issue and then merge the fork back into the parent collection when their Github PR is merged.
This is a decent solution, but for it to be a great solution, we need Jenkins to automatically fetch the forked postman collection and use that to run all the tests. The current setup requires the person who creates the collection fork to generate a PMAT key and enter it as credentials with a specific ID that has the Github PR number in it, and then Jenkins uses the PMAT key to fetch the collection fork.
The ideal solution would be to remove the need for the developer to do any kind of work in Jenkins and automate that part as well. If there was a way to fetch the collection fork using the label of the fork, that would be ideal. Then the developer would only need to create a fork and work there and Jenkins would automatically know that the fork was created and use that collection instead.
Is that possible at all?
Thanks in advance