Newman returns successful despite failures

Ask the Experts and Postman Tips
, ,
1

I have Jenkins running newman. Sometimes, I have some failures in assertions but it still returns a 0 return code (AKA succesful).

This is what it looks like in the console:

04:48:15 ┌─────────────────────────┬───────────────────────┬───────────────────────┐
04:48:15 │                         │              executed │                failed │
04:48:15 ├─────────────────────────┼───────────────────────┼───────────────────────┤
04:48:15 │              iterations │                     1 │                     0 │
04:48:15 ├─────────────────────────┼───────────────────────┼───────────────────────┤
04:48:15 │                requests │                    49 │                     0 │
04:48:15 ├─────────────────────────┼───────────────────────┼───────────────────────┤
04:48:15 │            test-scripts │                    96 │                     0 │
04:48:15 ├─────────────────────────┼───────────────────────┼───────────────────────┤
04:48:15 │      prerequest-scripts │                    52 │                     0 │
04:48:15 ├─────────────────────────┼───────────────────────┼───────────────────────┤
04:48:15 │              assertions │                    74 │                     3 │
04:48:15 ├─────────────────────────┴───────────────────────┴───────────────────────┤
04:48:15 │ total run duration: 16m 57.4s                                           │
04:48:15 ├─────────────────────────────────────────────────────────────────────────┤
04:48:15 │ total data received: 432.76kB (approx)                                  │
04:48:15 ├─────────────────────────────────────────────────────────────────────────┤
04:48:15 │ average response time: 20.6s [min: 159ms, max: 4m 32.2s, s.d.: 1m 4.4s] │
04:48:15 └─────────────────────────────────────────────────────────────────────────┘

Even though in the assertions row there are 3 failures, it still returns a 0 RC. Is there a way to have it return a non-0 ReturnCode? Some additional flag?

Thanks ahead!

2

This seems like a reasonable and simple solution (e.g.," have newman return a failure response when tests don’t pass).

I’m surprised that this request from 3 years ago was never answered.

If it’s worth anything, I would also appreciate this, as it would allow for better scripting with Newman.