Hello everyone!
I’m trying to put together a custom newman reporter which is able to generate a report and upload it on a remote service (specifically, into a Confluence page).
The problem I’m facing is that the upload itself is of course asynchronous, but I can’t find a way to make newman wait for the upload to finish before terminating the execution.
Ideally, the ‘done’ newman event should be triggered only after the upload has terminated, so that:
- when using newman as a library: the user can attach an handler to the ‘done’ event and be sure that any other action has completed
- when using newman in CLI: the user can be sure that any asynchronous operation has completed by the time the newman command terminates
As of today I’m observing that:
- when using newman as a library: the asynchronous upload terminates after the ‘done’ event gets fired
- when using newman in CLI: the upload is not executed at all (I don’t see the page uploaded to Confluence), I would guess because newman terminates before the asynchronous upload terminates.
I discussed this briefly with shamasis in this issue, and he confirmed (regarding the CLI) that node should wait for any asynchronous request to complete before terminate. Neverthless I still cannot see the page uploaded when running from CLI, although I’m sure that the upload itself works because if I run a script using newman as library I get the page uploaded, even if it gets there after the ‘done’ event.
I’m saying that the upload is completed after the ‘done’ event because of the outcome of the test case I added to the repo which is supposed to check that the async call gets fired: the (expected) error I get due to the fact that the test server is always returning an empty response is not thrown before the handler of the newman completion, but afterwards (in the afterEach() hook of the test).
You can check the code in the bugfix/upload-report branch, just run the test-integration test. Here is the output I get when I run it locally:
> [email protected] test-integration D:\workspaces\webstorm\newman-reporter-confluence
> node npm/test-integration.js
Running Integration tests using mocha and shelljs...
Installing newman & newman-reporter-confluence into the temp directory
Newman CLI
√ should correctly generate the report for a successful run (7665ms)
√ should correctly generate the report for a failed run (6792ms)
√ should correctly produce the report for a run with TypeError (7007ms)
√ should correctly produce the report for a run with one or more failed requests (6268ms)
1) should call the server for report upload
Newman Library
√ should correctly generate the report for a successful run (3891ms)
√ should correctly generate the report for a failed run (656ms)
√ should correctly produce the report for a run with AssertionError/TypeError (777ms)
√ should correctly produce the report for a run with one or more failed requests (435ms)
2) should call the server for report upload
3) "after each" hook
8 passing (42s)
3 failing
1) Newman CLI
should call the server for report upload:
Uncaught AssertionError: expected 0 to equal 1
+ expected - actual
-0
+1
at D:\workspaces\webstorm\newman-reporter-confluence\test\integration\cli.test.js:93:36
at D:\workspaces\webstorm\newman-reporter-confluence\node_modules\shelljs\src\exec.js:121:9
at ChildProcess.exithandler (child_process.js:280:7)
at maybeClose (internal/child_process.js:962:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:251:5)
2) Newman Library
should call the server for report upload:
Uncaught AssertionError: expected 0 to equal 1
+ expected - actual
-0
+1
at D:\workspaces\webstorm\newman-reporter-confluence\test\integration\library.test.js:101:36
at D:\workspaces\webstorm\newman-reporter-confluence\.temp\node_modules\newman\lib\run\index.js:308:25
at D:\workspaces\webstorm\newman-reporter-confluence\.temp\node_modules\newman\node_modules\async\internal\once.js:12:16
at iteratorCallback (.temp\node_modules\newman\node_modules\async\eachOf.js:60:13)
at D:\workspaces\webstorm\newman-reporter-confluence\.temp\node_modules\newman\node_modules\async\internal\onlyOnce.js:12:16
at D:\workspaces\webstorm\newman-reporter-confluence\.temp\node_modules\newman\node_modules\async\dist\async.js:473:16
at next (.temp\node_modules\newman\node_modules\async\dist\async.js:5329:29)
at D:\workspaces\webstorm\newman-reporter-confluence\.temp\node_modules\newman\node_modules\async\dist\async.js:969:16
at D:\workspaces\webstorm\newman-reporter-confluence\.temp\node_modules\newman\lib\run\export-file.js:41:13
at D:\workspaces\webstorm\newman-reporter-confluence\node_modules\nyc\node_modules\graceful-fs\graceful-fs.js:45:10
at FSReqWrap.oncomplete (fs.js:141:20)
3) Newman Library
"after each" hook:
Uncaught Error: Unexpected result form Confluence getContent API: {}
at confluence.getContentByPageTitle (.temp\node_modules\newman-reporter-confluence\lib\index.js:81:23)
at processCallback (.temp\node_modules\newman-reporter-confluence\node_modules\confluence-api\lib\confluence.js:52:9)
at D:\workspaces\webstorm\newman-reporter-confluence\.temp\node_modules\newman-reporter-confluence\node_modules\confluence-api\lib\confluence.js:156:13
at Request.callback (.temp\node_modules\newman-reporter-confluence\node_modules\superagent\lib\node\index.js:716:12)
at IncomingMessage.parser (.temp\node_modules\newman-reporter-confluence\node_modules\superagent\lib\node\index.js:916:18)
at endReadableNT (_stream_readable.js:1092:12)
at process._tickCallback (internal/process/next_tick.js:63:19)
----------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files | 0 | 0 | 0 | 0 | |
----------|----------|----------|----------|----------|-------------------|
=============================== Coverage summary ===============================
Statements : Unknown% ( 0/0 )
Branches : Unknown% ( 0/0 )
Functions : Unknown% ( 0/0 )
Lines : Unknown% ( 0/0 )
================================================================================
npm ERR! code ELIFECYCLE
npm ERR! errno 3
npm ERR! [email protected] test-integration: `node npm/test-integration.js`
npm ERR! Exit status 3
npm ERR!
npm ERR! Failed at the [email protected] test-integration script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
Process finished with exit code 3
It’s entirely possible that I’m missing something obvious (maybe a newman feature I don’t know or some internal js/node mechanism), but I’m starting to think that maybe this is just not possible with the current newman architecture.
Did someone already faced this problem?