Reporting Skipped Requests - pm.skipRequest() vs. pm.test.skip()

[jpjpjp]

In this thread, I described how I have a test collection that includes pre-request scripts that may call pm.execution.skipRequest() if certain required variables are not set.

In these circumstances, I want my collection to continue to run, but ideally, there would be some programmatic output that would make it easy for me to identify if requests that were skipped. In the referenced thread, I asked if there was a way to add a param to the pm.execution.skipRequest() call to have it trigger the logic that would increment the skipped test count in newman or runner.

Danny helpfully explained that skipping a request is different than skipping a test, which makes sense. I’m curious if anyone has any advice on how I could programmatically detect skipped tests.

One idea I had was to write some console.error output in these circumstances and redirect newman’s stderr output to a file, but it seems that newman is intercepting this (it adds some red to the console.error output).

In an ideal world I’d love to see a column “Skipped” added to the newman output, ie:
┌─────────────────────────┬──────────────┬────────────┐────────────┐
│ │ executed │ failed │ skipped │
├─────────────────────────┼──────────────┼────────────┤────────────┤
│ iterations │ 1 │ 0 │ 0 │
├─────────────────────────┼──────────────┼────────────┤────────────┤
│ requests │ 47 │ 0 │ 3 │
├─────────────────────────┼──────────────┼────────────┤────────────┤
│ test-scripts │ 47 │ 0 │ 12 │
├─────────────────────────┼──────────────┼────────────┤────────────┤
│ prerequest-scripts │ 17 │ 0 │ 0 │
├─────────────────────────┼──────────────┼────────────┤────────────┤
│ assertions │ 357 │ 0 │ 0 │
├─────────────────────────┴──────────────┴────────────┤────────────┤

I realize this doesn’t lend itself to programmatic inspection, but it would make it easy to realize that tests were skipped at a glance. I haven’t played much with newman’s built-in reporters, so if anyone has had luck with this using a different reporter I’d love to hear about that!

Thanks in advance for any suggestions!

[danny-dainton]

Hey @jpjpjp

If you wanted to give htmlextra a go, that has a separate tab for skipped tests.

newman-reporter-htmlextra

[jpjpjp]

Thanks, Danny, for your response. I gave htmlextra a try, but the output doesn’t capture skipped requests nor count the tests that were omitted because pm.skipRequest() was called in a pre-request script as skipped.

My goal is really to capture skipped requests.

Whenever I call pm.skipRequest(), I always log something with console.error, and I had hoped to redirect stderr to an output file and post-process that. Unfortunately, newman seems to redirect stderr back to stdout (if using the cli reporter), or completely absorbs it if not using the cli reporter.

Is it possible to ask newman not to do this?

[danny-dainton]

I didn’t mention that it would capture the skipped requests - It will show the number of tests that were skipped in a request that uses pm.test.skip in a script.