HTML report showing cumulative data in per-execution assertions

🙋 Help
1

When I use newman at the CLI and use the HTML reporter with the standard report, I noticed that the block for each detailed request shows the global (cumulative) numbers in the Tests table, not the numbers for the assertions associated with that single request.

I looked at the HTML template, and the problem seems to be in the following code (line 144 sq.):

{{#if assertions.length}}
    <div class="col-md-12">&nbsp;</div>
    <div class="col-md-4">Tests</div>
    <div class="col-md-8">
         <table class="table table-responsive table-condensed">
              <thead><tr><th>Name</th><th>Pass count</th><th>Fail count</th></tr></thead>
              <tbody>{{#each assertions}}<tr><td>{{this.name}}</td><td>{{this.passed}}</td><td>{{this.failed}}</td></tr>{{/each}}</tbody>
        </table>
    </div>
{{/if}}

It looks like assertions.passed and assertions.failed might be the cumulative numbers (essentially the same as cumulativeTests.failed and cumulativeTests.failed). I’m not sure how to debug this, and the sample data provided at https://github.com/postmanlabs/newman/blob/develop/examples/reports/html-reporter-template-data.json doesn’t show the substructure of the executions[i].assertions object

2

@Wabiloo This is by design, as the HTML reporter aggregates statistics by default. However, a lot of users have expressed a similar question. However, a lot of users have asked a similar question to the one above, so we might look at changing this behaviour in the future. In the meantime, you could look into writing your own custom Newman reporter, or use one of the existing ones: https://www.npmjs.com/search?q=newman-reporter.

3

I’ve been working on this new reporter to try and help with some of the issues in the original HTML reporter.

Happy to talk through anything @Wabiloo and change some things :grin:

1 Like