How can I extract object/key names from nested and varying JSON payload

New to APIs/Postman
1

Hi all,
I am C++ programmer - trying to get to grips with Postman, JSON, REST APIs etc.

I am trying to write a Postman test to visualize some JSON response data - which I would like to show in a table format with the key names as column headings.

The problem is that the number of data items in a response can vary and the key names can also vary - depending on the input parameters.

Details (like screenshots):
So say, for the following JSON which has two data items:

{
    "data": [
        {
            "input": [
                {
                    "value": "ABC",
                    "identifierType": "a1"
                }
            ],
            "output": [
                {
                    "value": "BT",
                    "identifierType": "b1",
                    "name": "BT GROUP",
                    "status": "Active",
                    "classification": "Ordinary"
                }
            ]
        },
        {
            "input": [
                {
                    "value": "BCD",
                    "identifierType": "a1"
                }
            ],
            "output": [
                {
                    "value": "EFG",
                    "identifierType": "b1",
                    "name": "SIEMENS",
                    "status": "Active",
                    "classification": "Ordinary"
                }
            ]
        }
    ]
}

I can get the values such as ABC, EFG, a1,b1, Ordinary etc - but I want to get the names such as ‘input->value’, ‘input->identifierType’,‘output->value’, ‘output->identifierType’, ‘output->status’

I am using something like the following to get the values in my parseData method

const results = response.map(elem => (
        {
            inputValue: elem.input[0].value,
            inputIDType: elem.input[0].identifierType,
            ouputValue: elem.output[0].value,
            outputIDType: elem.output[0].identifierType,
            outputName:  elem.output[0].name
        } 
    ));

which is fine, but I can’t figure out how to do something similar for the key names.
Just to be clear, in any single request and JSON response, all the input and output key names will be the same for all the entries. If I make a different request with different input parameters, then it is possible that the key names in that response will be different.

thanks in advance for any tips/help.

2

Update.
So, I got the following working - on the assumption that there will always be an input and output object:
const headers = ;
for (const key in response[0].input[0]){
headers.push(Input ${key})
}
for (const key in response[0].output[0]){
headers.push(Output ${key})
}

so that I get:
[“Input value”, “Input identifierType”, “Output value”, “Output identifierType”, “Output name”, …]

But what if the input and output objects are called something else?

3

Have you considered using the visualiser?

4

Hi,
Not sure what you mean by the visualizer? As I mentioned I am pretty new to Postman.

with the return values from the above parseData method, I am then calling:

pm.visualizer.set(template, {
                // Template will receive stringified JSON
                results: JSON.stringify(parseData(JDict.data)[0]),
                headers: JSON.stringify(parseData(JDict.data)[1])
            });

and that gives me the output I want…

but not sure how to generalize the code, so that it still works if the input and output nodes are called something else?

5

Can you do something like:

for (const _outer in response[0]) {
   for (const _inner in _outer) {
      for (const key in _inner) {
          headers.push(${_inner} ${key})
      }
   }
}
6

Data Visualization for APIs | Postman Visualizer

7

I don’t really see how this would work.

You sort of need to know what the objects are called, so you can create headers, etc.

The other option is to visualise all of the data which means you should be able to create headers based on the keys being returned, but that doesn’t sound like its what you want.

You can create a new object to visualise against, so it only includes the keys you are interested in, but again, you need to know which keys, so I can’t see how this would work if you don’t know the keys or have a consistent response.

The following seems to work against your data, but it does require you to know the elements in the response.

let response = pm.response.json().data

var template = `

    <style type="text/css">
        .tftable {font-size:14px;color:#333333;width:100%;border-width: 1px;border-color: #87ceeb;border-collapse: collapse;}
        .tftable th {font-size:18px;background-color:#87ceeb;border-width: 1px;padding: 8px;border-style: solid;border-color: #87ceeb;text-align:left;position:sticky;top:0;}
        .tftable tr {background-color:#ffffff;}
        .tftable td {font-size:14px;border-width: 1px;padding: 8px;border-style: solid;border-color: #87ceeb;}
        .tftable tr:hover {background-color:#e0ffff;}
    </style>
    
    <table class="tftable" border="1">

        <tr>
            <th>Input Value</th>
            <th>Input Identifier Type</th>
            <th>Output Value</th>
            <th>Output Identifier Type</th>
            <th>Output Name</th>
        </tr>
        
        {{#each response}}

            <tr>

                {{#each input}}
                    <td>{{value}}</td>
                    <td>{{identifierType}}</td>
                {{/each}}

                {{#each output}}
                    <td>{{value}}</td>
                    <td>{{identifierType}}</td>
                    <td>{{name}}</td>
                {{/each}}
                
            </tr>

        {{/each}}

    </table>

`;

console.log(response);

pm.visualizer.set(template, {
    response: response
});

image
image796×245 11.8 KB

8

Thanks for all the replies - much appreciated.

@david-franklin - your answer partially works with a minor change…
for (const _outer in response[0]) {
for (const _inner in _outer) {
for (const key in _inner) {
headers.push(${_inner} ${key})
}
}
}
gives headers as [“0 0”, “1 0”, “2 0”, “3 0”, “4 0”, …]
but if I change it to the headers.push($*{_outer*} ${key})
then I get [“input 0”, “input 0”, “input 0”, “input 0”, “input 0”, …]
so, the outer name is being extracted but not the innermost key…

@michaelderekjones - thanks for the link, but I thought I was already using the visualiser?

And with regards to your most recent post, I found the following Visualize any JSON/CSV as a table! - Community showcase - Postman - which seems to be able to extract all the node names and use them to create column headings. But as a newbie, I cannot figure out where/how in the code it is doing the extraction…

9

That is generating a HTML table from the response. (With a fair bit of JavaScript).

The problem with this is that it will visualise all of the data, not just the keys that you had in your original example.

I don’t see how you can just pull out just the value, identifierType and name without knowing what those keys are called.

10

Thanks - no worries.

I just thought that since the following get me half way - without knowing the names such as identifierType etc:

    for (const key in response[0].input[0]){
        headers.push(`Input ${key}`)
    }
    for (const key in response[0].output[0]){
        headers.push(`Output ${key}`)
    }

and gives me:
0: “Input value”
1: “Input identifierType”
2: “Output value”
3: “Output identifierType”
4: “Output name”
5: “Output status”
6: “Output classification”

and @david-franklin 's code also allows me to extract the outer ‘input’ and ‘output’ names, I just figured I could somehow combine the two and get the same headers as above - albeit without having to explicitly specify response[0].*input[0] and response[0].output[0]

Thanks all.

11

@unlseg

let headers = []; 

for (const outerkey in response[0]){
    for (const innerkey in response[0][`${outerkey}`][0]){
        headers.push(`${outerkey} ${innerkey}`)
    }
}

console.log(headers);

image
image767×152 7.66 KB

1 Like
12

Woohoo! :tada:

Thanks @michaelderekjones for teaching an old dog new tricks! :+1:
I did try some stuff with the $ and outerkey - but clearly was not getting the syntax right.

1 Like
13

Update, I thought I would share the changes I made for the actual data extraction code as well.
Previously I was explicitly specifying the key names to get the actual data values:

const results = response.map(elem => (
        {
            inputValue: elem.input[0].value,
            inputIDType: elem.input[0].identifierType,
            ouputValue: elem.output[0].value,
            outputIDType: elem.output[0].identifierType,
            outputName:  elem.output[0].name
        } 
    ));

Which did not achieve my objective of a general-purpose visualizer, regardless of the key names, etc.

So, I replaced the above with the following - applying some of the learning here from @michaelderekjones and @david-franklin - thanks!

function parseData(response, host) {
    const results = response.map(function(val, index){
        temp = [];
        for (const outerkey in val){
            for (const innerkey in val[`${outerkey}`][0]){
                temp.push(val[`${outerkey}`][0][`${innerkey}`]);
            }
        }
        return temp
    }
    );
    
    const headers = [];

    for (const outerkey in response[0]){
        for (const innerkey in response[0][`${outerkey}`][0]){
            headers.push(`${outerkey} ${innerkey}`)
        }
    }
    return [results, headers]
}

Just sharing this in case it helps someone else…

1 Like
14

@unsleg

Have a look at the following. (Just something to consider).

let response = pm.response.json().data.data

let headers = []; 
for (const outerkey in response[0]){
    for (const innerkey in response[0][`${outerkey}`][0]){
        let obj = {title: `${outerkey} ${innerkey}`}
        headers.push(obj);
    }
}
// console.log(headers);

let results = [];
for (const record in response){
    let obj = {}
    let cellNumber = 1;
    for (const outerkey in response[record]){
        for (const innerkey in response[record][`${outerkey}`][0]){
            let value = response[record][`${outerkey}`][0][`${innerkey}`];
            // obj[`${outerkey} ${innerkey}`] = value
            obj[`cell${cellNumber}`] = value;
            cellNumber++;
        }
    }
    results.push(obj);
}
// console.log(results);

var template = `

    <style type="text/css">
        .tftable {font-size:14px;color:#333333;width:100%;border-width: 1px;border-color: #87ceeb;border-collapse: collapse;}
        .tftable th {font-size:18px;background-color:#87ceeb;border-width: 1px;padding: 8px;border-style: solid;border-color: #87ceeb;text-align:left;position:sticky;top:0;}
        .tftable tr {background-color:#ffffff;}
        .tftable td {font-size:14px;border-width: 1px;padding: 8px;border-style: solid;border-color: #87ceeb;}
        .tftable tr:hover {background-color:#e0ffff;}
    </style>
    
    <table class="tftable" border="1">

        <tr>
            {{#each header}}
                <th>{{title}}</th>
            {{/each}}
        </tr>

        {{#each result}}
            <tr>
                <td>{{cell1}}</td>
                <td>{{cell2}}</td>
                <td>{{cell3}}</td>
                <td>{{cell4}}</td>
                <td>{{cell5}}</td>
                <td>{{cell6}}</td>
                <td>{{cell7}}</td>
            </tr>
        {{/each}}
      
    </table>

`;

pm.visualizer.set(template, {
    result: results,
    header: headers
});

image
image2071×147 4.97 KB

This will never be that all purpose, as it relies on a similar structure. It needs your outer keys to be arrays and your inner keys to be values. Any changes to that and it will break.

To format it nicely in the table, you need to know the name of the key, but I just set each key to cell1, cell2, etc. (Hence its quite fragile).

15

Thanks, @michaelderekjones for that reply and code snippet.
Unfortunately, when I use it in my query I get the error:
TypeError: Cannot read properties of undefined (reading ‘0’)
I will try and investigate this when I have time - but at present, I cannot.
I have a working solution as per my earlier post - so will stick with that for now.
Cheers.

16

@unlseg

Remove the second .data (this is just how Postman Echo returns the data).