Asserting for undefined is not working as expected

[bebemau]

Hello everyone, I have tried and tried and tried but it just wont work, am I missing something?

My test simplified:

var responseBody = pm.response.json();
var price =  responseBody.price;

pm.test("Status code is 200", function () {
     pm.response.to.have.status(200);
});

pm.test("price exists", function () {
    console.log("checking if price exists");

    if (pm.expect(price).not.eq(undefined)) {
        console.log("price exists");
    }
    else {
        console.log("price does not exists");
    }
});

Response:

{
    "productId": "DP1234"
}

I will expect “price does not exists” to be logged, but nothing is logged except for “checking if price exists”. What am I doing wrong??

For the record, i have tried:

pm.expect(price).to.be.not.undefined;
pm.expect(price).not.to.be.undefined;
pm.expect(price).to.be.a('number');

I have also tried setting price to 0 and assert if not zero:

var price =  responseBody.price;

They all did the same thing. It almost seems like it is ignoring all the code after the assertion. I have run out of ideas…

Thanks in advance for any feedback!

[michaelderekjones (Mike Jones)]

I added a price element to the response and this appears to work.

const response = pm.response.json()

let price = response.price;
console.log(price); // undefined

pm.test("price to be undefined", function () {
    pm.expect(price).to.be.undefined
});

Removed the price element and the test passes.

Another way of doing this is to use the Object.keys function.

pm.test("price doesn't exist", function () {
    pm.expect(Object.keys(response)).to.not.include('price')
});

It’s also worth looking at the conversation here.

arrays - Checking if a key exists in a JavaScript object? - Stack Overflow

As the recommended way of doing this is…

pm.test("price doesn't exist v3", function () {
    pm.expect(!("price" in response)).to.be.true
});

[bebemau]

@michaelderekjones

pm.expect(Object.keys(response)).to.not.include('price')

Thank you @michaelderekjones.

Is the logic i had incorrect? I mean it almost seems like a bug to me that postman will ignore all the code after an assertion.

I shouldnt need to add the element in the response to get my test working, because the use case is the exact oppsite where that element does not exist.

i did try pm.expect(Object.keys(response)).to.not.include(‘price’) and exact same behavior.

[michaelderekjones (Mike Jones)]

Postman uses Chai for its assertion library.

It works on a hard fail approach.

So as soon as an assertion fails, it immediately fails the pm.test block and stops executing any more code.

That is just how Chai works. It’s philosophy is that soft assertions shouldn’t be used. A test either passes or it doesn’t. So there is no point in executing any more code after an assertion fails.

Not sure what you mean by “adding the element to get the test working”.

If you want to check that price doesn’t exist, you are going to need to have the word price somewhere in the assertion.

I recommend checking the link on Stack overflow where they discuss the best way to determine if a key exist in an object or array which is…

!("price" in response)

This is different to checking if a key exists but is null or undefined.

In your example, it looks like you are checking for the existence of the key.

[bebemau]

Hi @michaelderekjones, thanks for the explanation.

@michaelderekjones

Not sure what you mean by “adding the element to get the test working”.

I was just responding to the comment “I added a price element to the response and this appears to work.” in your previous post. And that is true, if the element exists, then my logic works, but that is not what I wanted to test for, I want to test when the key does not exist in the response.

I guess I was trying to avoid having two checks. The actual logic is if price does not exist set a collection variable. If as soon as a test fails it does not run any subsequent logic, the only way i can think of to do that will look like code below. Do you have any recommendation how to achieve that without having two separate checks for price, one if statement, and another as a test? It just seems silly. Thanks again for your feedback!

var responseBody = pm.response.json();
var price = responseBody.price;

pm.test(“price exists”, function () {
console.log(“checking if price exists”);

if(price == undefined) {
//code to set collection variable
}

pm.expect(price).not.eq(undefined)) ;
});

[michaelderekjones (Mike Jones)]

Testing something in the response and then setting a variable are technically two different activities.

I guess what you are trying to do is put this into a single test block, which may or may not be the right approach.

Let’s go back to what you want to test again.

You said you want to test when the key does not exist in the response.

Should it pass or fail that test?

Your latest code has the test passing if price exists which is not testing that the key does not exist. Quite the opposite.

Therefore if you want to set a collection variable if the price doesn’t exist, then this block of code should be outside of the test block checking if price exists. Just move the IF statement outside of the test.

If you really do want to test that price doesn’t exist, then this means you can go back to having the code for updating the collection variable in the test block.

On a side note, you need to be in control of your test data.

The response should either always return a price element, or it shouldn’t and you would test accordingly. You shouldn’t need an IF statement.

If you want to test both outcomes, then it should be two separate requests as you can’t test both outcomes in a single request.

[bebemau]

Sorry for the confusion.

The expected response looks like this (simplified), with this response all tests should pass:

{
“price”: 1234,
“productId”: “DP1234”
}

To do a negative test, the code I have in mind is to set the variable price to response.price, and test the variable.

If the variable price is undefined, set collection variable, if not undefined, run other tests as an example if productId exists etc.

Figure i would just state the goal instead of putting code

I am pretty new at this so please let me know what would be the recommended approach. Thank you!

[michaelderekjones (Mike Jones)]

If your expected response includes the price, then you need to test that price exists.

If it doesn’t, then the test should fail (as expected).

When testing, you should always try and make your test fail to ensure you are not getting a false positive.

If you can’t get your API to return a response without the price by the test data you send, then this can be done by parsing the response and then removing the element from the response variable before you run your test.

You create your test and make it pass. You remove the element from the response and test it again to ensure it fails, and you then set it back to passing. This gives you a level of confidence that your test is correct.

This is NOT a negative test. It’s just testing that you are not getting false positive.

With negative testing, you are trying to break the product with the expected result being a failure of some sort. You then test for the failure. But the principal is the same, it should always fail when you send the request.

You can’t mix the positive and negative tests. They need to be separate requests with appropriate tests in each.

For example, if you send the wrong data types and the server response with a 5xx status code. You would be testing that the status code is not a success. 2xx.

Another example is if you had an authentication step, you would send the wrong password and ensure you get a 401 unauthorised error. With negative testing, you should also be testing that the API produces reasonable error messages, and doesn’t just crash if you send incorrect data.

If you are always expecting x number keys in your response. Then you might want to test the returned JSONSchema to ensure those keys are available. This also isn’t a negative test, and should always pass if the expected keys are in the response and will fail if its missing any of the keys.

The Postman Learning Centre should explain how to do JSONSchema validation, so please have a look there first.

I’m just not seeing the need for an IF statement, and I’m also not sure what you are going to do with the collection variable you set.

[michaelderekjones (Mike Jones)]

Here are three ways to test the keys in a response.

If you want to test the values, then you will need separate tests for that.

// https://jsonschema.net/

let response = pm.response.json().data; 

// method 1 - to.have.property

let keysToCheck = ["price", "productId"];

keysToCheck.forEach(key => {
    pm.test(`${key} exists in response V1`, () => {
        pm.expect(response).to.have.property(key);
    });
});

// method 2 - object keys & to.include

let keys = Object.keys(response);

keysToCheck.forEach(keyToCheck => {
    pm.test(`${keyToCheck} exists in response V2`, () => {
        pm.expect(keys).to.include(keyToCheck);
    });
});

// method 3 - JSON Schema.

const schema = {
    'type': 'object',
    'properties': {
        'price': {
            type: 'number'
        },
        'productId': {
            type: 'string'
        }
    },
    required: ['price', 'productId'],
    additionalProperties: false
};

pm.test('API response schema is valid', () => {
    pm.expect(response).to.have.jsonSchema(schema);
});

[bebemau]

I am good with how to test for “price” since postman is giving me the expected result. My only question is how to do the if, without testing twice with an if statement and a pm.test statement. Maybe the answer is i have to do both, that’s what i am trying to find out.

Again the response is simplified there are other properties in the returned json. If price exist and non zero then i want to also test if the other properties exists etc, otherwise no point to. If price does not exist or zero, set collection variable so the subsequent requests do not run.

Thanks!

[michaelderekjones (Mike Jones)]

You could replicate the pattern that Postman uses for most of its courses.

Which is to define a counter like the following.

// counter for passed tests
let pass = 0
let totalToPass = 3

In your test block after all of the assertions, you up the number for pass.

pass += 1

This will only trigger if all of the assertions have passed.

Then at the end of your code, you have an IF statement checking the status of the pass variable.

if (pass == totalToPass) {
   console.log("pass");
} else {
   console.log("fail";
   // set collection variable or
  // pm.setNextRequest("null") 
}

If you want to stop executing requests if it fails, I would use setNextRequest(“null”) rather than a collection variable.

If you want to test for price first, then the other attributes, you can put this all in the same test block. If price doesn’t exist, it will immediately fail the test and not check the other attributes.

I would recommend being careful with this. I would advise to have the test check a single attribute. As soon as the first attribute fails, the whole test will fail, which means you don’t get the feedback on the other elements. Therefore it might be price and another element that is not being returned, but you wouldn’t neccessarily know until you fix the first element.

However, I’m going to point out again that you need to control your test data. Your response should either have price or it shouldn’t. This type of dynamic testing is fragile. Everything above is just a workaround to not having solid test data and consistent responses.

[bebemau]

I see, that makes sense. Thank you @michaelderekjones that was very helpful!