How to validate requests against swagger (OpenAPI 2.0) schema

šŸŒ± Just Getting Started
, ,
1

Iā€™m probably being dense here, but please bear with me.

My question:

I have created an API, by importing the following Swagger/OpenAPI 2.0 file:
https://raw.githubusercontent.com/tmforum-apis/TMF641_ServiceOrder/master/TMF641-ServiceOrdering-v4.1.0.swagger.json

It imported just fine: I allowed it to import as both collection + API.

However, when I create a client request in the Collection, the request is not validated against the Swagger schema (e.g. I can add nonsense top-level fields, or wrong data types for fields, and they are happily sent out).

Equally, if I create a Mock endpoint, the received request is not validated either. It just tells me it canā€™t find any matching mock. Of course, I can create matching mocks for specific valid requests, but I would like requests which are not valid according to the schema to be rejected.

Am I missing something obvious here? It seems to be a simple enough requirement, that if Iā€™ve loaded an OpenAPI schema, that Postman can tell me whether the request Iā€™ve prepared in the client collection is valid or not.

Iā€™ve already tried:

I did find a blog post about validating responses, using the ā€œTestsā€ feature. It suggested I added the following under the API ā€œTestsā€ tab:

pm.test("Validate schema", () => {
            pm.response.to.have.jsonSchema(schema);
});

But as far as I can see from the manual, tests are only run after the response has been received.

So instead, I tried creating a pre-request script:

pm.request.to.have.jsonSchema(schema);

This tells me that ā€˜schemaā€™ is not defined. So I added above this:

var schema={
    "swagger": "2.0",
    "info": {
        "title": "API ServiceOrdering",
...snip entire large swagger doc...
};

This runs, but the (invalid) request is still happily sent out. I changed it to:

console.log("XXX:" + pm.request.to.have.jsonSchema(schema));

and I get XXX:[object Object] in the console.

If I change it to

console.log("XXX:" + pm.request.to.have.jsonSchema(schema).keys().join());

then I get ā€œAssertionError: keys requiredā€. As this point, I suspect I donā€™t know enough about the syntax of this ā€œfluentā€ testing API.

Itā€™s also quite likely that Iā€™m misunderstanding the difference between JSON schema and Swagger/OpenAPI.

If this is a FAQ, or in the documentation please do point me at it - but Iā€™ve done quite a lot of searching already and couldnā€™t find anything. Thereā€™s plenty of documentation about how to validate the Swagger API document itself, as opposed to validating requests and responses against the Swagger API doc. But I donā€™t need to validate the Swagger API doc: this is already provided to me in its final state and I wonā€™t be changing it.

Many thanks for taking the time to read this!

2

Hi @candlerb

I think that you are passing the OpenAPI spec as the schema you want to validate against, which is not quite right.

The schema validation is to validate the JSON object and would include types that you want to match against (eg: object vs array or integer vs string).

Here is an example I put together, hope this helpsā€¦

3

Thank you, but your example shows this JSON schema that youā€™ve written:

const schema = {
    "type": "object",
    "properties": {
        "id": {
      "type": "integer"
    },
    "category": {
      "type": "object",
      "properties": {
        "id": {
          "type": "integer"
        },
        "name": {
          "type": "string"
        }
      },
      "required": [
        "id",
        "name"
      ]
    },
    "name": {
      "type": "string"
    },
    "photoUrls": {
      "type": "array",
      "items": [
        {
          "type": "string"
        }
      ]
    },
    "tags": {
      "type": "array",
      "items": [
        {
          "type": "object",
          "properties": {
            "id": {
              "type": "integer"
            },
            "name": {
              "type": "string"
            }
          },
          "required": [
            "id",
            "name"
          ]
        }
      ]
    },
    "status": {
      "type": "string"
    }
  },
  "required": [
    "id",
    "category",
    "name",
    "photoUrls",
    "tags",
    "status"
  ]
};

pm.test("Validating the API schema", () => {
    pm.response.to.have.jsonSchema(schema);
});

And it shows you validating an API response against this schema.

So, I think maybe Iā€™ve havenā€™t explained clearly enough what Iā€™m looking for.

  1. I have an existing Swagger / OpenAPI 2.0 spec that I want to use (not a JSON schema, although I understand there are similarities)
  2. I want to validate that the request that Iā€™ve prepared, not the response, complies with that spec.

That is: I want to check my request is compliant to that existing OpenAPI spec at the URL I gave before.

And as a secondary point, Iā€™d like the Mock server to check that any request it receives is compliant to that schema too.

Either of these would achieve my goal: if I actually had to post a request to the Mock server to have it validated, that would be fine too.

Does that make sense?

4

I think if you import the OpenAPI spec into Postman when you create requests it automatically validates it against the spec ā€¦ I may be wrong, @arlem may be able to provide a more detailed answer though :slight_smile:
(Hope you donā€™t mind me tagging you here Arlemi).

5

I did import the OpenAPI spec. The requests are not being validated.

6

Thanks for tagging me @w4dd325!

@candlerb We do have request validation (cf. API Builder | The Exploratory - YouTube) but thereā€™s a few pre-requisites:

  • it needs to be enabled in your Settings (see ā€œRequest Validationā€)
  • the collection needs to have been generated, and still linked from the API, my recommendation would be to first import the API spec, then add a collection to it to make sure they are properly linked, docs for this are here
  • I need to check this one, but there is a chance request validation is only available from OpenAPI 3.0 and above, Iā€™m checking with the team and Iā€™ll let you knowā€¦
7

Thank you for your reply.

This is a fresh install of Postman 10.12.10 on macOS. Settings are all at defaults, and Iā€™ve checked that Request Validation was already set to ON.

I did import the API spec, and selected the option ā€œOpenAPI 2.0 with a Postman Collectionā€ (i.e. it imported the swagger API and created a collection at the same time)

image

However, if validation only works with OpenAPI 3.0, that would explain it :slight_smile:

If that turns out to be the issue, Iā€™d like to know if thereā€™s any workaround, e.g. using the Pre-Request script hook.

In fact, even if you know of any web site which can validate an API request against the corresponding OpenAPI 2.0, that would be sufficient. Itā€™s just that Iā€™ve been having trouble locating such a service, and so I had hoped Postman would have this functionality built-in.

I found plenty of tools for validating JSON docs against JSON Schema, plenty of tools for validating the OpenAPI spec itself (i.e. the JSON file which defines the API). But finding a tool to validate the actual requests against the OpenAPI spec is what Iā€™m struggling with!