Testing Properties and Arrays correctly in GET

[LiamC (Liam Cooper)]

Hi, I just want to make sure that this is correctly testing that An Object and an Object within an array is present.

Example api:

{
  "items": [
    {
      "oID": "0000001211",
      "oInvoiceNo": "1234567",
      "OrderBlocks": [
        {
          "olLine": 1,
          "olProductCode": "001"   
        }
      ]
    }
  ]
}

Example test:

pm.test("Order Details are Present", () => {
    pm.response.to.have.jsonBody("oID");
    pm.response.to.have.jsonBody("oInvoiceNo");
    pm.response.to.have.jsonBody("OrderBlocks[0].olLine");
    pm.response.to.have.jsonBody("OrderBlocks[0].olProductCode");

});

It’s returning as a Pass, but I’ve been experimenting with syntax all day with loads of different tests(I can’t remember .js and pm.* so I just refactor constantly until it works), and I’m sure I had one earlier that brought back a False Pass, just wanted to make sure that this is a correct way for checking that the correct properties, and properties within arrays are present in a response.

I also don’t completely understand why this seems to work, shouldn’t “oID” also require me to have an array like (“items[0].oID”) ? Yet when I try this, it fails to find oID.

Was trying this initially and it failed, so the above is my new “fix”:

pm.test("Order Details are Present", () => {
     pm.response.to.have.jsonBody("items.OrderBlocks.olLine");
});

Any help/advice on testing this kind of stuff correctly is really appreciated, thanks!

[danny-dainton]

Hey @LiamC,

I’m not sure what your overall goal is or what you would like to test but you could try something like this to check the types and keys of that response.

pm.test("Order Details are Present", () => {
    let jsonData = pm.response.json()
    pm.expect(jsonData.items).to.be.an("array");
    pm.expect(jsonData.items[0]).to.have.keys('oID','oInvoiceNo','OrderBlocks').and.be.an("object");
    pm.expect(jsonData.items[0].OrderBlocks).to.be.an("array");
    pm.expect(jsonData.items[0].OrderBlocks[0]).to.have.keys('olLine','olProductCode').and.be.an("object");
});

It’s a bit messy and it will only ever check the first object in those arrays. Lots of different ways that you could check/test that response just all depends on what you want to test for I guess.

Have you considered using some form of schema validation as a different option?

[LiamC (Liam Cooper)]

@danny-dainton ! Oh wow, I’ve had your “All Things Postman” stuff open for the past 2 days trying to wrap my head around Postman. Crazy.

I think what I’m testing is probably made redundant by using:

const jsonData = pm.response.json();
const schema = JSON.parse(environment.testSchema)

pm.test("Schema is Valid", () => {
    pm.expect(tv4.validate(jsonData, schema)).to.be.true;
});

And having the API as ‘testSchema’ in Environment Variables.

I’m just trying to check that every response has the expected fields returned that I can see on the Swagger documentation of the API.

I was going to try and get something like forEach to work next, if I could, with pm.expects inside the function? For multiple ‘items’ arrays being returned.

jsonData.items.forEach(item => {
   pm.expect(jsonData.items[0]).to.have.keys('oID','oInvoiceNo','OrderBlocks')
});

But, maybe Schema validation is already checking all of this.
If I wanted to test if an object beyond the 1st one, was present in an array, how would that look?

I need to keep reading up more.
Your write-ups have been a huge help actually! I’m just greener than green at the moment so I’m probably getting confused!

[danny-dainton]

Awesome! Thanks for checking that out - It’s very old now though and is falling behind the current functionality that we have introduced since then

You should be able to just use this syntax to get the schema from the variable:

const schema = pm.environment.get("testSchema")

What does your schema look like?

In terms of looping through the data, you could modify the test you wrote to check each object in the items array. I’ve taken the same response data and added another object with one of the keys different from what you’re checking against, just to see this fail.

If you add this code to your Tests tab, you should be able to see the failed test:

var sampleData = {
  "items": [
    {
      "oID": "0000001211",
      "oInvoiceNo": "1234567",
      "OrderBlocks": [
        {
          "olLine": 1,
          "olProductCode": "001"   
        }
      ]
    },
    {
      "wrongKey": "0000001212",
      "oInvoiceNo": "1234568",
      "OrderBlocks": [
        {
          "olLine": 2,
          "olProductCode": "002"   
        }
      ]
    }
  ]
}

pm.test("Debugging Test - Object Key are Correct", () => {
    sampleData.items.forEach(item => {
       pm.expect(item).to.have.keys('oID','oInvoiceNo','OrderBlocks')
    });
})

[LiamC (Liam Cooper)]

Oh, I think I get the syntax now!

That Debug test example is pretty much exactly what I needed to understand, I think I was close-ish.

That’s just an example schema above, the one I’m actually working from belongs to the company I work for unfortunately, as much as I’d love to share it to get your insight on how to approach testing a brand new in-development API and what type of tests an experienced tester would do. In fact, this is all I’m doing at the moment - just trying to create a test case as robust as possible for the current proof of concept API I’ve been given, beyond 200 status, objects present and values match correct types/regex.

Regarding the debug test example, how would you go about automating these kinds of tests in a Collection Run? I feel like it would need engineering to fail but retrieve a “pass” test status so you can still integrate it with something like Jenkins, right?

Otherwise, a failed test that you intend to fail, would flag the API as not correct, when it’s functioning exactly how it should.

Thanks again!

EDIT: Can’t get tv4.validate to work either. I can set a variable to anything and it still returns as Pass. I don’t understand how to get tv4.validate to actually use the Schema. Example that should fail but doesn’t:

const = jsonData = pm.response.json();
var schema = "not real schema";
pm.test("Schema is Valid", () => {
    let validation = tv4.validate(jsonData, schema);
    if (validation !== true) {
        console.log(tv4.error);
    }
    pm.expect(validation).to.be.true;
});

I’ve tried reading through the tv4.validate documentation and I’m just not getting it, should I be copying the Schema into the Tests Tab to reference as you did in your above Debug Test example?.

[danny-dainton]

I would actually recommend using the ajv validation module instead - You can see an example of this in working in this other thread:

Schema validation ignoring array sub-fields type

Hey @fermani Welcome to the community Not sure what was happening for you locally but the JSON Schema you posted wasn’t valid JSON - It was missing a closing " at the end of the access key. I’ve used a different Schema validation module that’s built-in to Postman, to help with this question. It’s very similar to the tv4 module but is actively maintained. I mocked out the response body from the question and this seems to be working (failing the test due to the wrong data type). var…

[LiamC (Liam Cooper)]

I’ve copied the exact code in that link to, and changed the “id” to “string” instead of “integer” and no matter what I try, it will always pass.

I’m stumped.

[danny-dainton]

You’ve copied the code and run it against your response?

That’s checking a different schema though, that user has a response like this:

{
    "access": [
        {
            "id": 1,
            "client": "127.0.0.1",
            "apikey": "apikey1",
            "description": "Local call"
        }
    ]
}

That wouldn’t work for your schema as the structure is different, it was more of an example of the structure.

You could use something like this:

var Ajv = require('ajv'),
    ajv = new Ajv({ logger: console, allErrors: true }),
    schema =  {
    "type": "object",
    "properties": {
      "items": {
        "type": "array",
       "items": {
         "type": "object",
         "required": [
             "oID",
             "oInvoiceNo",
             "OrderBlocks"
         ],
         "properties": {
           "oID": {
             "type": "string"
           },
           "oInvoiceNo": {
             "type": "string"
           },
           "OrderBlocks": {
             "type": "array",
             "items": {
               "type": "object",
               "required": [
                 "olLine",
                 "olProductCode"
               ],
               "properties": {
                 "olLine": {
                   "type": "integer"
                 },
                 "olProductCode": {
                   "type": "string"
                 }
               }
             }
           }
         }
       }
     }
   }
}

pm.test('Schema is valid', function() {
    pm.expect(ajv.validate(schema, pm.response.json()), JSON.stringify(ajv.errors)).to.be.true;
});

This should also log out all the schema errors so you’re not just getting a generic error message from the test result.

I suck at creating schemas, this is a site that I use to help generate them from a JSON response.

JSON Schema Generator

[Manoj_Mittal]

Hi All,

I also need some help to create some test cases where my url is fetching data in such form:

Group:[
{
“element”:{
“id”: “ABC”,
“name: :ABC”;
“ref”: 1
},
“rateplan”:[
{
“element”:{
“id”: “SAM”,
“name”: “XYZ”,
“ref”: 1000
},
“duration”:{
“type”: “Month”,
“amount”: 36
};
“category”: “Standard”,
“type”: “Postpaid”,
"price: [
{
“type”: “Month”,
“amount”: 1000,
},

there are various layers of data within elements.

any help is appreciated. I tried the examples provided above but i am getting syntax error.

thanks!!

[danny-dainton]

Hey @Manoj_Mittal

Welcome to the community!

Would you be able to create a separate topic for this question please. I don’t want it to get lost within this topic thread.

[Manoj_Mittal]

started a new topic, thanks!!

[vdespa]

@LiamC writing schemas that work is not only an advanced topic but it is not really well supported in Postman.

My advice would be to stay away from schema validation, unless you REALLY know what you are doing.

Here are a few reasons:

• writing and maintaining proper schemas is time consuming
• there is no dedicated way to edit / save them and they tend to be quite big
• just following a “paste here your response and I will generate something for you” tool rarely works
• you can easily be fooled by “never failing schemas” (as you have already noticed
• if later a schema validation does fail, you will have a hard time figuring out what has changed (at least with the tv4 lib)

Maybe the new ajv lib in Postman does a better job than the tv4 lib.

So consider if schema validation is something that you want to invest some time or if you can simply live with testing properties one-by-one.

Good luck!

[LiamC (Liam Cooper)]

I managed to get something to work, after banging my head against a wall for hours.
I do agree this schema validation stuff may be a lot more than I can handle at the moment, so I may go back to what I had originally.

It’s been a learning experience though!

If you’re interested by the way, this is what I found would actually test Passes & fails correctly:

const jsonData = pm.response.json();

pm.test('Schema is valid', function() {
    var Ajv = require('ajv');
    ajv = new Ajv(),
    schema = { //Schema should contain an Array
        "type": "array",
        "items": {//In this Array there should be a Properties Object
                    "type": "object", //These are the properties within this object. This will test TYPE of Property.
                    "properties": {
                        "gameId": { "type": "integer" },
                        "name": { "type": "string" },
                        "description": { "type": "string" },
                        "minPlayers": { "type": "integer", "maximum": 1},
                        "maxPlayers": { "type": "integer", "maximum": 1},
                        "rating": {"type": "string" },
                        "online": {"type": "boolean" }
                        
            },"required" : ["gameId", "name", "description", "minPlayers", "maxPlayers", "rating", "online"]//Required will make sure that the Properties listed are present
        }
    };
    
    var validate = ajv.compile(schema);
    
    var valid = validate(jsonData)
    if(valid === false){
        console.log('Invalid: ' + ajv.errorsText(validate.errors));
    }
    
    pm.expect(valid).to.be.true;
  
});

Thank you

[danny-dainton]

Glad you got a solution that works for you and learnt some new skill along the way.

It feels like the outcome changed slightly from the originally posted response data - Looks like a different endpoint is being checked now

[LiamC (Liam Cooper)]

Yeah, another example api I had set up for Crud testing haha.

[bpricilla (Pricilla B)]

Hi @ks.suhas12996 Welcome to community

Can you please create a new topic and provide the details? Also please make sure to provide all the details, so that we can help you better

Please read the link on how to ask your queries in the forum !!

[danny-dainton]

What validations are you wanting to add - @bpricilla doesn’t know your context.

• What part of that data set is important to you?
• What would you want to check?
• Is is just the structure of the JSON?
• The values of certain/all fields?
• The datatypes?
• Something else?

Adding a response body and saying that you would like to “put some test case validations” doesn’t really give anybody anything to go off

[bpricilla (Pricilla B)]

Thanks @danny-dainton @ks.suhas12996 Sorry I almost lost track of this post. As Danny mentioned, kindly list down your requirements

[ks.suhas12996 (KS Suhas)]

Hi @danny-dainton Need to validate the below things:

1. Structure of JSON
2. values of all fields
3. and also how we can test both conditions not being null and not being undefined

@bpricilla Can you please help me on this?

[alberto-att (Alberto)]

Hi, could you help me on how to validate for a certain property value when I have multiple objects?

I can test the first object, however since the data could contain 100 of object I need to identify against the property value.
For example if
seller is Toyota the price should be null
seller is VW price can be null or not null
if Model is >2000 price must not be null

let carData = pm.response.json();
pm.test(“testArray”, function () {
pm.expect(carData[0]).to.have.property(‘car’);
pm.expect(carData[0].price).to.be.null;
});

[
{
Car: ‘Golf’,
Model : 2000
Seller: ‘VW’,
Price: ‘null’
}
{
Car: ‘Camry’,
Model : 2000
Seller: ‘Toyota’,
Price: ‘10000’
}
{
Car: ‘Corola’,
Model : 2020
Seller: 'Toyota,
Price: ‘20000’
}
]

Thanks in advance