I’m trying to set a custom message ONLY when status code is not 200 but also check for network errors. Therefore, whenever I have a response status code that is not 200, I want to output the response body. If there are any network errors, just fail as usual. Having below code I assume:
If response status code is not 200, then output the http response as assertion error
Test any network issues (socket/connreset/etc) AND also response code to not be 4XX or 5XX
It should be OK but I am worried if any network issues, I don’t want it to try to print the response or have an test exception for something that its not there. Just the standard error from the pm.response.to.not.be.error.
pm.test("Status code is 200", function () {
pm.expect(200,'HTTP Error Response -> \'' + pm.response.text() + '\'').to.be.eq(pm.response.code);
pm.response.to.not.be.error;
});
The purpose of it is to improve the integration tests so the tip for the error is right there in the tests logs so that I don’t have to manually check the app logs to find out since its just prints the status code in the standard way.
Maybe I am missing some concepts but I could’t find much documentation around it. is this a good test or any way to improve it?
I would suspect that network errors would also fail to produce an 200 status code, in which case it will never hit the pm.response.to.not.be.error.
The pm.test is a hard fail. If you have multiple assertions, as soon as the first failure happens, the test automatically fails and the rest of the code is not executed. (Postman uses Chai under the hood for assertions and it doesn’t have the concept of soft assertions).
You might need to wrap the test in an IF statement where you check for this “network error” first although I’m not sure you can determine sockets/econnreset this way. If you don’t have a response, I don’t think there will be anything to target like there would be for 4xx, 5xx errors.
I mean “network errors” as broader range checking any kind of error condition beyond just the HTTP status code (DSN resolution, SSL handshake failures, etc).
Yes, I know it would also fail to produce 200 status code in case of network errors. On this case, I want it to fail as usual by using pm.response.to.not.be.error and do not try to print the http response body.
I don’t know nor found a documentation saying exactly what pm.response.to.not.be.error is really checking for. Also, the official documentation also give us an example with three assertions:
In all cases, my only concern with the provided solution is that it doesn’t break the test itself while trying to expect the first check. To exemplify, let’s assume these cases:
No network errors & status 200 → Test case (OK)
No network errors & status 500 → Test fails and prints the response body (OK)
Network error & unknown status → Will pm.expect break the test itself? Since it might not have a status code nor response text, I don’t want it to try to print the response.
The thing is: If I put the pm.response.to.not.be.error I can guarantee a broader errors to be catch there but it ALSO catches status code. Whenever I have a status code (or response from API) I want to print the response. Otherwise just print the error itself.
I mean print by the test output itself. It cannot be in the log since this is part of integrated test and the postman log wont be populated to the output.