Code coverage with Postman /Newman

🙋 Help
, , ,
1

I have test cases written in postman for testing APIs
I use Newman command line integration for running my collection.

I just got a new requirement for code coverage now.
I want to to know is there any way or any integration by which I can know my code coverage without rewriting whole tests??

PS: I have my code in nodeJs.

2

I think you need to question what should the Postman tests cover. Just because it is a requirement, does not necessarily mean it makes sense and it is a good idea.

Postman does not interact with your code, it interacts with the API that the application exposes. As Postman does not have access to the code directly, it cannot report back which code was executed.

Code coverage is usually a requirement that applies to unit testing or similar tests that are executed directly on the code.

Postman is an API testing tool, it implicitly tests how the entire system made of components works together.

So unit testing / API testing (or integration testing) are two different things with different purposes.

3

@vdespa
I know Postman is tool for API testing and doesn’t interact with code.
That is why I have this question that “Is there any tool/framework where I can just export my Postman collections and then integrate that with my source code”?

I

4

@vdespa I have a question. So the Postman Test that is written with either jest, mocha or chia. How do you create a type of coverage that will send back the results of what you tested. Just by grabing that information over the test you covered. I am trying to do something similar but I want to create a code coverage that not looking at the direct code but just the coverage of what my test is stating I am testing? Is that possible?

5

A Postman test is written using tests which contain assertions. The most of the assertions are using the ChaiJS library.

At this level, we can only discuss of coverage in terms of (for example) which endpoints, methods and status codes are covered. For this, you first need to have a way of knowing this in advance and a way to track which aspects were tested.

While is it possible to use the new API tab and to provide a OpenAPI schema, it is not possible to track the coverage.

2 Likes
6

@vdespa thank you for this response.

7

This is definitely possible. It’s a weird requirement for sure but I ran into a scenario where I wanted this.

Working in a legacy project where (for reasons) the test suite is essentially postman and I needed a way to get coverage reports so I could get a grasp of what lines in these 1k+ controller methods are even being hit by tests.

So here’s what worked for me.

  1. Run the API using nyc: yarn nyc node server.js for example.

  2. Run the tests (doesn’t matter if it’s newman or in my case a suite of tests that run in a separate application and hit the API).

  3. Quit the server.

  4. Convert the nyc output JSON to an html report. By default this command will work:

npx nyc report --reporter html --reporter text -t .nyc_output --report-dir coverage/summary

Now you have coverage reports!

Downside of this is that any code executed during the initial bootstrapping of your application will look like it has coverage too. But for my purposes that’s a downside I can live with.

8

I found this on GitHub:
https://community.postman.com/t/code-coverage-with-postman-newman/7708/5

I find the tool very helpful when it comes to covering API testing.
However, it is not yet available in a final version, but it already works