Hi,
I am using the contract testing functionality in the API section. I have create a collection to test and I have imported the OpenAPI definition but when I click on validate I always receive Validation OK, even if i use a mock to force an invalid json response that does not match with the schema definition.
Do you have any idea?
thanks a lot
Hi @Moon, Iām also having this issue and wondering if you solved it. Seems like the contract testing tests some parts but not the validity of the response body. I can copy and past the schema into the javascript tests and validate it with tv4 just fine, but that sort of diminishes the value here.
The documentation seems to indicate support for this feature: https://learning.postman.com/docs/postman/design-and-develop-apis/validating-elements-against-schema/
Hard to believe such an obvious feature would be missing.
There is a whole lot of validation going on so it can be confusing to follow each step. This validation is making sure the collection you generated from your OpenAPI still in alignment. This validation isnāt making sure the response of each API is valid, it is for making sure OpenAPI contract is met, not the individual API schema contract. Does that make sense? We have a lot of work to do in defining the right words to use for this process. Try changing a path name in your collection, and you should see it not validate.
That make sense.
Thanks a lot for your help!
Hi @Moon
I have created a PR on the OpenApi-to-Postman library,that would allow you to generate your Postman collection including several Postman tests .
The goal of this PR is to facilitate contract testing via Postman, based on your based on your openapi specification.
Currently the following Postman tests are generated automatically:
Perhaps this will solve your request with regards to the contract testing.
Feel free to give feedback on the PR.
Just out of curiosity, why doesnāt it validate the response schema is valid as well?
Iāve been using AJV for response schema validation, but it seems like itās essentially just re-writing (with some differences) whatās in the OpenAPI definition.
That with some differences part might be more difficult than you think, @robmcec 
Iāve pointed you at this before, but Iāll reference it again because you seem to be doing a lot of manual work, but I created a Postman collection that will automatically generate contract tests off of your open api spec and validate response codes and response schema. You are more then welcome to use it as-is or piece together what i did for the automatic response validation.
Hi Allen, thanks for the link.
If memory serves, the api spec needs to have certain requirements for the generator to work.
Iām not the spec designer and the current one is still in the process of developing the basic endpoints.
If I have some time, Iāll create a new collection, copy our current spec and try the test generator.
Those schemaās are not something Iāll want to be manually editing if we implement contract testing. Thatās for sure.
Thanks!
@robmcec The test suite generation PR New Feature - Generate tests from OpenAPI / Swagger - automated contract testing #56 by thim81 Ā· Pull Request #225 Ā· postmanlabs/openapi-to-postman Ā· GitHub will re-use the OpenAPI schema to generate a āpm.response.to.have.jsonSchemaā test.
// Test whether the response matches the schema
pm.test("[PUT] /lists/v2/audiences/{audienceId}/contacts/{contactId} - Schema is valid", function() {
pm.response.to.have.jsonSchema(schema,{unknownFormats: ["int32", "int64"]});
});