For example, in the link above, this tests if https://api.coindesk.com…… can get the bitcoin exchange rate. This doesn’t appear to be testing that the current code that’s being deployed by travis is working as expected. Since postman will be testing HTTP requests, in order to test the current code, the current code needs a URL that postman/newman can run the tests against right? How can we do this?
Our current setup, is during the travis deployment, we deploy our code to another lambda stage, which we already know the urls of, and these are preset within postman. Therefore when travis is deploying it runs the tests against the code we have deployed during the travis deployment. But this means deploying code twice and seems way overkill.
So my question is, how can we use postman to test current code in CI/CD pipelines?
To be honest, I did not get the example showed in the post with “Integration with Travis CI” and I think your observation is valid.
Without seeing the Travis pipeline, it is hard to tell what the issue is.
The CI/CD workflow can look something like this:
Build > Unit Tests > Deployment > API tests
So as you can see, you first need to deploy your code somewhere and after this, you can run the Postman tests. If the address where this is deployed is dynamic, you can pass it between the build stages and give it to newman at runtime.
Depending on the application architecture, sometimes you can simply start the server exposing the API locally before the Postman tests start.
This workflow, API tests running after deployment, is what we are trying to avoid. We would like the deployment to depend on the success of the API tests, just like the unit tests.
Since we are using serverless, I did run sls offline during the travis pipeline (before deployment) and run the tests against that (postman URL was localhost + port we added in serverless yaml), which was too hacky. Had to run sls offline and then detach from that command wait until its started up, run the tests, then try and get the exit code so we could stop the deployment if it was 1.
We are using lambdas (node) and we have already tried starting the server locally during deployment using sls offline but we could not get the exit code from the server after the tests had passed / failed as mentioned above. Has anyone else managed to overcome this issue when using lambdas?