Newman tv4 schema validation does not work

Ask the Experts and Postman Tips
1

Problem: My newman tests are working fine except schema validation using tv4.
Here is the error. It says missing required property QIS. However the response body has “QIS”.

Validation errors:

17:00:59.508

    • Object:{}
      • message:“Missing required property: QIS”
      • name:“ValidationError”
      • type:“Error”

Here is the code snippet:

 pm.test("Test 18: Response body is valid", function () {
   console.log({
     schema1: globals['search_response_schema']
   });

   data = JSON.parse(responseBody);

   var schema_response = JSON.parse(globals['search_response_schema'])
   console.log("schema_response2 =: ", schema_response)

   //Reference: https://stackoverflow.com/questions/48098530/how-do-i-test-json
   valid = tv4.validate(data, schema_response, true, true);
   tv4.validateResult(request, globals['search_response_schema']).valid;

   console.log("Validation errors: ", tv4.error);
   if (valid !== true) {
     console.log(tv4.error);
   }

   pm.expect(valid).to.be.true;
 });

Note: On the console it is logging the schema correctly

2

Hi @rjayaram, this issue cannot be triaged by just using this code snippet.
Can you share a sample (retract any sensitive information) of your response body and a sample of the schema (retract any sensitive information) for us to test this script and then maybe the community will be able to help you better?

Thanks!

1 Like
3

Hi Sivcan

Thanks for looking into this. As I could not attach the sample file here I am sending the sample schema and response body pasted into a document and attaching to your email.

Thanks

4

Sivcan

Thanks for looking into this. As I could not attach the sample file in community site, I am attaching a PDF with the sample schema and response body pasted into a document.

(Attachment PDF_APR10_Postman_Schema_Validation_Error.pdf is missing)

5

Hi

Both email and this community thread rejected a PDF document attachment. Sending the response body and schema in PNG format.

PNG1_responseBody
PNG1_responseBody1700×2200 16.3 KB
!

Pasted schema below from pre-req script as I could not attach:

var search_response_schema =
{
“definitions”: {},
“$schema”: “http://json-schema.org/draft-07/schema#”,
“$id”: “http://example.com/root.json”,
“type”: “object”,
“title”: “The Root Schema”,
“required”: [
“timing”,
“data”
],
“properties”: {
“timing”: {
“$id”: “#/properties/timing”,
“type”: “array”,
“title”: “The Timing Schema”,
“items”: {
“$id”: “#/properties/timing/items”,
“type”: “object”,
“title”: “The Items Schema”,
“required”: [
“QIS”
],
“properties”: {
“QIS”: {
“$id”: “#/properties/timing/items/properties/QIS”,
“type”: “object”,
“title”: “The Qis Schema”,
“required”: [
“duration”,
“reqId”
],
“properties”: {
“duration”: {
“$id”: “#/properties/timing/items/properties/QIS/properties/duration”,
“type”: “integer”,
“title”: “The Duration Schema”,
“default”: 0,
“examples”: [
385
]
},
“reqId”: {
“$id”: “#/properties/timing/items/properties/QIS/properties/reqId”,
“type”: “string”,
“title”: “The Reqid Schema”,
“default”: “”,
“examples”: [
“6c845d80-5a23-11e9-bc8f-25dc5895059c”
],
“pattern”: “^(.)$"
}
}
}
}
}
},
“data”: {
“$id”: “#/properties/data”,
“type”: “array”,
“title”: “The Data Schema”,
“items”: {
“$id”: “#/properties/data/items”,
“type”: “object”,
“title”: “The Items Schema”,
“required”: [
“id”,
“type”,
“attributes”
],
“properties”: {
“id”: {
“$id”: “#/properties/data/items/properties/id”,
“type”: “string”,
“title”: “The Id Schema”,
“default”: “”,
“examples”: [
“apps”
],
“pattern”: "^(.)$”
},
“type”: {
“$id”: “#/properties/data/items/properties/type”,
“type”: “string”,
“title”: “The Type Schema”,
“default”: “”,
“examples”: [
“source”
],
“pattern”: “^(.)$"
},
“attributes”: {
“$id”: “#/properties/data/items/properties/attributes”,
“type”: “object”,
“title”: “The Attributes Schema”,
“required”: [
“total_count”,
“results”
],
“properties”: {
“total_count”: {
“$id”: “#/properties/data/items/properties/attributes/properties/total_count”,
“type”: “integer”,
“title”: “The Total_count Schema”,
“default”: 0,
“examples”: [
1
]
},
“results”: {
“$id”: “#/properties/data/items/properties/attributes/properties/results”,
“type”: “array”,
“title”: “The Results Schema”,
“items”: {
“$id”: “#/properties/data/items/properties/attributes/properties/results/items”,
“type”: “object”,
“title”: “The Items Schema”,
“required”: [
“attributes”
],
“properties”: {
“attributes”: {
“$id”: “#/properties/data/items/properties/attributes/properties/results/items/properties/attributes”,
“type”: “object”,
“title”: “The Attributes Schema”,
“required”: [
“app”,
“app_array”
],
“properties”: {
“app”: {
“$id”: “#/properties/data/items/properties/attributes/properties/results/items/properties/attributes/properties/app”,
“type”: “string”,
“title”: “The App Schema”,
“default”: “”,
“examples”: [
“Photoshop”
],
“pattern”: "^(.)$”
},
“app_array”: {
“$id”: “#/properties/data/items/properties/attributes/properties/results/items/properties/attributes/properties/app_array”,
“type”: “array”,
“title”: “The App_array Schema”,
“items”: {
“$id”: “#/properties/data/items/properties/attributes/properties/results/items/properties/attributes/properties/app_array/items”,
“type”: “object”,
“title”: “The Items Schema”,
“required”: [
“score”,
“name”
],
“properties”: {
“score”: {
“$id”: “#/properties/data/items/properties/attributes/properties/results/items/properties/attributes/properties/app_array/items/properties/score”,
“type”: “integer”,
“title”: “The Score Schema”,
“default”: 0,
“examples”: [
1
]
},
“name”: {
“$id”: “#/properties/data/items/properties/attributes/properties/results/items/properties/attributes/properties/app_array/items/properties/name”,
“type”: “string”,
“title”: “The Name Schema”,
“default”: “”,
“examples”: [
“Photoshop”
],
“pattern”: “^(.*)$”
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
};

7

I am sorry, but PNG cannot be used to debug this issue.
Will be needing the response JSON to actually test out the script.

You can just type six backticks (`) and paste a sample response in b/w them here instead of uploading a pdf file.
Also, make sure it’s the correct sample response and a valid JSON.

Please refer the following on how to format code blocks: https://help.github.com/en/articles/creating-and-highlighting-code-blocks

You can check if your response JSON is valid or not here: https://jsonlint.com/

Thanks!

8

Hi Sivan

I tried pasting the response body within backticks. It gives it exceeds the limit. Let me know where I can paste the response body.

Regards

9

You can just paste it here: https://codeshare.io/G6MOng

Make sure it’s just an example and not the complete response body.
Also, please check that it’s valid JSON.

10

Done. Pl remove the response from the above link after you copy for debugging.

11

@rjayaram I saw that the response body doesn’t have the QIS property, that’s at least what the error was thrown.

For eg (one of the errors):
At the beginning of the response, it says that sparkTemplates has the required property QIS missing.
So, I added a random QIS property to the sparkTemplates object and then the error went away. But yes, there are more validation errors that came after that.
image

You can update your script like so to debug even better:

pm.test("Test 18: Response body is valid", function () {
   data = resp;

   var schema_response = JSON.parse(globals['search_response_schema']);
   console.log({ schema_response });
   
   // More detailed error, so that you can see where the validation fails
   tv4.setErrorReporter(function (error, data, schema) {
    return { e: "Error: " + error.code, data };
   });

   //Reference: https://stackoverflow.com/questions/48098530/how-do-i-test-json
   valid = tv4.validate(data, schema_response, true, true);

   console.log({ valid });
   console.log("Validation errors: ", tv4.error);
   if (valid !== true) {
     console.log(tv4.error);
   }

   pm.expect(valid).to.be.true;
});

Also, you can open the DevTools for an easier view of the console logs, also with the modified script that I wrote above that gives more detailed error message, you can check that where exactly the validation is failing (on which piece of data).
You can click on View > Developer > Show DevTools (Current View)

image
image770×285 112 KB

It shows that the response has some issues / the schema has some issues.
If you think there’s no issue there, then it looks like an issue with the tv4 library itself.

You can locally setup a node project and verify with tv4, whether it’s actually working or not.
tv4 repo: GitHub - geraintluff/tv4: Tiny Validator for JSON Schema v4

12

Hi

As you can see from my response schema and actual response ,“QIS” is present. I think it is an issue with tv4 library itself as some people mentioned in stackoverflow. could you suggest an efficient way / tool to validate schema with postman newman scripts.

Thanks.

13

Hi Postman team

I am following up on my earlier query. Please suggest a better schema validation library.
As I mentioned earlier there are no issues with the schema. I see some posts on stackoverflow that tv4 has issues.

Thanks, Rjayaram

14

You can use ajv which is also built into postman.

AJV Docs - https://github.com/epoberezkin/ajv

You’ll have to require ajv in the test script.

15

Hi Sivcan

Thanks. could you share some postman tests example using ajv for schema validation.

Thx

16

Example:

var Ajv = require('ajv'),
    ajv = new Ajv({ logger: console }),
    schema = {
        "properties": {
            "alpha": {
                "type": "boolean"
            }
        }
    };

pm.test('Schema is valid', function() {
    pm.expect(ajv.validate(schema, { alpha: true })).to.be.true;
    pm.expect(ajv.validate(schema, { alpha: 123 })).to.be.false;
});
1 Like
17

@singhsivcan … Thanks for Ajv snippet, its been helpful to some point, but please if you will indulged me , am trying to validate a schema response body of a post with with ajv and its passing even though it should fail … i can see in your snippet that you drill down a to “alpha” property before validation.

How do i validate the response body of a post with the predefined schema.

Thanks in advance

18

If you could share a sample response and the schema with me, I’ll try to debug it on my end.
Currently, I can’t guess what might be going wrong.

Thanks!

19 
 {
    "fl": [
        {
            "fl_Ref": "test_test",
            "dimension": [
                {
                    "dimensionTypeValue": "SE",
                    "dimensionTypeRef": "PD",
                    "dimensionTypeValueRef": "PA"
                },
                {
                    "dimensionTypeValue": "Corporate ",
                    "dimensionTypeRef": "OR",
                    "dimensionTypeValueRef": "CB"
                }
            ],
            "fl_Key": "5cd1a2947a3c4",
            "fl_Name": "test_test",
            "links": []
        }
    ]
}

apologies i couldn’t get sample response in well formatted code view …am not too familiar with the features in the forum :frowning_face:

21

I wanted both, the response and the schema that you’ve written.

22

@singhsivcan thanks in advance for your response and assistance below is the schema
Schema == >

let Ajv = require('ajv'),
  ajv = new Ajv({ logger: console }), 
  schema = {
    "title": "FlConfiguration",
    "description": "Configuration for a Fl",
    "version": "1-0-3",
    "type": "object",
    "additionalProperties": false,
    "required": [
        "flRef",
        "flName"
    ],
    "properties": {
        "flRef": {
            "description": "Fl reference - external key for underlying configuration.  Also used to auto-generate target references",
            "type": "string",
            "pattern": "^[a-zA-Z0-9_-]{0,10}$"
        },
        "flKey": {
            "description": "Internal key for Tag within underlying datastore ",
            "type": "string"
        },
        "flName": {
            "description": "Name of the flow",
            "type": "string"
        },
        "flBuilderVersion": {
            "description": "Version of component used to generate the fl",
            "type": "string"
        },
        "sourceType": {
            "description": "Underlying Source Type",
            "enum": [
                "Refer",
                "Inter",
                "Exter",
                "Syst"
            ]
        },
        "dimension": {
            "description": "Dimensions",
            "type": "array",
            "items": {
                "type": "object",
                "required": [
                    "dimensionTypeRef",
                    "dimensionTypeValueRef"
                ],
                "properties": {
                    "dimensionTypeRef": {
                        "description": "Reference for underlying Dimension type",
                        "type": "string"
                    },
                    "dimensionTypeValueRef": {
                        "description": "Reference for value associated with underlying Dimension Type",
                        "type": "string"
                    },
                    "dimensionTypeValue": {
                        "description": "Dimension value",
                        "type": "string"
                    }
                }
            }
        },
        "lanes": {
            "description": "Swim lanes for fl visuals",
            "type": "array",
            "items": {
                "type": "object"
            }
        },
        "steps": {
            "description": "Process steps within the fl visuals",
            "type": "array",
            "items": {
                "type": "object"
            }
        },
        "links": {
            "description": "Links between process steps",
            "type": "array",
            "items": {
                "type": "object"
            }
        },
        "maps": {
            "description": "(Top-level) fl maps",
            "type": "array",
            "items": {
                "type": "object"
            }
        }
    }
}

Response ==>

{
    "fl": [
        {
            "fl_Ref": "test_test",
            "dimension": [
                {
                    "dimensionTypeValue": "SE",
                    "dimensionTypeRef": "PD",
                    "dimensionTypeValueRef": "PA"
                },
                {
                    "dimensionTypeValue": "Corporate ",
                    "dimensionTypeRef": "OR",
                    "dimensionTypeValueRef": "CB"
                }
            ],
            "fl_Key": "5cd1a2947a3c4",
            "fl_Name": "test_test",
            "links": []
        }
    ]
}