Running collections

We’re currently using postman to test our endpoints. We have multiple collections and environments. Our API uses a JWT token for security.

We’d like to run our collections and report on the results outside of the postman UI.

I’m currently using (or trying to) a newman library script (and running it via nodejs) such that it triggers the collection and spits out an htmlextra report.

I’m having a very difficult time getting this to work correctly as the newman library doesn’t have access to the variables in postman and we have to export the enviornment json file, but we don’t keep our API key in the “initial value” of the environment variables. This means I have to copy\paste the jwt token into the environment json.

The script is turning into a mix of getpostman api calls and an exported json file with the jwt token copy and pasted into it.

So not maintainable or easy by any means.

What is the easiest way to accomplish what we’re trying to do?

-Thanks

BTW, here’s an example of the script.

require('dotenv').config();
const { Console } = require('console');
const newman = require('newman'); // require newman in your project
const key = process.env.postman_api_key;

 // The signature for the run function is newman.run(options: object , callback: function) => run: 
EventEmitter
newman.run(
{
    //collection: require('.\\Orders_1.postman_collection.json'),
	collection: 'https://api.getpostman.com/collections/13204628-74a6e824-c8d1-45fd-83e8-8a9003304b37?apikey=' + key,
    environment: require('.\\localHost.postman_environment.json'),
    //environment: 'https://api.getpostman.com/environments/12415606-e03e06c1-00f4-422a-891a-4e6984aa7cf3?apikey=' + key,
	reporters: 'htmlextra', // htmlextra is a reporting package for newman that will require installlation using NPM.
    reporter: 
	{ //Go here for documentation: https://www.npmjs.com/package/newman-reporter-htmlextra
		htmlextra: //Report options
		{
			export: '.\\Results.html', //Directory and name
			template: '.\\InnovaAPITemplate.hbs', //Template used for report. Html, CSS and JS are all embedded.
			title: 'Innovations API',
			showEnviornmentData: true,
			browserTitle: "Innovations API",
			showMarkdownLinks: true,                
			showFolderDescription: true,
            //skipSensitiveData: true //Hides request header because it can contain the authorization tokens.               
		}
	}
},// <-- End of options object
function (err,summary)// <-- Beginning of callback function
{
	if (err)
	{ 
		console.log(err);
		throw err; 
    } else 
    {
        //console.log(summary);
    }
}// <-- End of callback	
 ).on('start', function (err, args) { // <-- start event. 
	console.log('running a collection...');
}).on('done', function (err, summary) { // <-- done event.
if (err || summary.error) {
    console.error('\nCollection run encountered an error.');
}
else {
    console.log('\nCollection run completed.');
 }
})
;

Hey @robmcec

How are you creating that JWT? Is it was a request in the app or using the Auth Helper?

Maybe it’s external to the app altogether:smile: - Have you looked into to creating a request in the Pre-request Script to create this and not have to copy and paste things between Postman and Newman.

I fully understand the Newman part of what you’re doing I just can’t visually connect the JWT fetching part in my head - Can you share an example of that process?

Thank you for the reply, here you go.
We have an endpoint that retrieves the jwt token and sets an environment variable.

Here’s the steps.
We execute the Get request /auth. There’s a test script that then sets an environment variable called “jwtToken”.

If I work in PM everything is fine because the env variable is set and all the requests use it. So when I run a request using the collection runner everything is sunshine and rainbows.

BUT when I try to run a collection using the newman library, it has no way of knowing what that jwt variable is.

So I can’t run collections unless I provide newman with that jwt variable that holds the token. The only way I’ve been told to do that is by exporting the environment json file. BUT that only exports the Initial Value of that variable and not the current value. The initial value isn’t set by the script only the current value.

Finally, I have to copy and paste the jwt token value into the export and then run the script.

Although, I’m still getting the error getaddrinfo ENOTFOUND {{host}}{{version}}

Which doesn’t give me a lot to go on.

Is there a better way to do this?

Is there anything else you’re using in that token request that could be impacting things? Do you set a username / password on that request to get the token?

If you’re able to run that in the app, I can’t think of what would be blocking that…without seeing it for myself.

Have you tried logging out the environment variables using for the run in another on event like you have in your script?

It should be in summary.environment / summary.globals I think.

There is a user name and password that gets the token. It’s hardcoded into our /auth authorization for now.

Once it’s retrieved though, it’s just an env variable.

How do I log an environment variable in the newman library if I don’t have don’t have access to it?

I can log the contents of the environment json, but that doesn’t really tell me if Newman isn’t handling it correctly. Correct?

UPDATE: I logged the summary.environment and summary.globals.
Like this:
console.log("summary.environment is: " + summary.environment);
console.log("summary.globals is: " + summary.globals);

running a collection…
Created the htmlextra report in this location: .\Results.html
summary.environment is: [object Object]
summary.globals is: [object Object]

They both return [object Object].

Please forgive my ignorance, I’m a newb! :slight_smile:

You would need to wrap that in JSON.stringify()

I’m really confused by all this and it’s make it even more difficult for me because I can’t see the collection of the way you’re running it. :pensive:

If the collection contains that request to get the Auth token and save that as an environment variable, there shouldn’t be a need to manually copy and paste that anywhere.

Newman would just to the same as what the Collection runner is currently doing.

I agree. I can’t attach the collection, or I would.

Here’s the /auth endpoint if that helps.

{
	"info": {
		"_postman_id": "74a6e824-c8d1-45fd-83e8-8a9003304b37",
		"name": "InnovaAPI - QA",
		"description": "API for Innovations Lab Management System",
		"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
	},
	"item": [
		{
			"name": "Authentication",
			"item": [
				{
					"name": "/auth",
					"event": [
						{
							"listen": "test",
							"script": {
								"id": "7edf7254-1258-4f1e-8ef1-dac2c4448925",
								"exec": [
									"var jsonData = JSON.parse(responseBody);\r",
									"pm.environment.set(\"jwtToken\", jsonData.data.jwtToken);"
								],
								"type": "text/javascript"
							}
						}
					],
					"request": {
						"auth": {
							"type": "basic",
							"basic": [
								{
									"key": "password",
									"value": "{{innoPass}}",
									"type": "string"
								},
								{
									"key": "username",
									"value": "{{innoUser}}",
									"type": "string"
								}
							]
						},
						"method": "GET",
						"header": [
							{
								"key": "x-api-key",
								"value": "API_KEY"
							}
						],
						"url": {
							"raw": "{{baseUrl}}/auth",
							"host": [
								"{{baseUrl}}"
							],
							"path": [
								"auth"
							]
						},
						"description": "Authentication which returns a jwtToken"
					},
					"response": [
						{
							"name": "OK",
							"originalRequest": {
								"method": "GET",
								"header": [
									{
										"description": {
											"content": "Added as a part of security scheme: basic",
											"type": "text/plain"
										},
										"key": "Authorization",
										"value": "Basic <credentials>"
									}
								],
								"url": {
									"raw": "{{baseUrl}}/auth",
									"host": [
										"{{baseUrl}}"
									],
									"path": [
										"auth"
									]
								}
							},
							"status": "OK",
							"code": 200,
							"_postman_previewlanguage": "Text",
							"header": [
								{
									"key": "Content-Type",
									"value": "application/json"
								}
							],
							"cookie": [],
							"body": "{\n \"data\": {\n  \"authenticationToken\": {\n   \"Expires\": \"2020-10-29T10:50:37Z\",\n   \"jwtToken\": \"jwtToken\"\n  }\n }\n}"
						},
						{
							"name": "BAD REQUEST",
							"originalRequest": {
								"method": "GET",
								"header": [
									{
										"description": {
											"content": "Added as a part of security scheme: basic",
											"type": "text/plain"
										},
										"key": "Authorization",
										"value": "Basic <credentials>"
									}
								],
								"url": {
									"raw": "{{baseUrl}}/auth",
									"host": [
										"{{baseUrl}}"
									],
									"path": [
										"auth"
									]
								}
							},
							"status": "Bad Request",
							"code": 400,
							"_postman_previewlanguage": "Text",
							"header": [
								{
									"key": "Content-Type",
									"value": "application/json"
								}
							],
							"cookie": [],
							"body": "{\n \"error\": {\n  \"message\": \"error message\",\n  \"code\": -30911017\n }\n}"
						}
					]
				}
			],
			"protocolProfileBehavior": {}
		}

I’m using the following script:

require('dotenv').config();
const { Console } = require('console');
const newman = require('newman'); // require newman in your project
const key = process.env.postman_api_key;

// The signature for the run function is newman.run(options: object , callback: function) => run: EventEmitter
newman.run(
	{
        //collection: require('.\\Orders_1.postman_collection.json'),
		collection: 'https://api.getpostman.com/collections/13204628-74a6e824-c8d1-45fd-83e8-8a9003304b37?apikey=' + key,
        //environment: require('.\\localHost.postman_environment.json'),
        environment: 'https://api.getpostman.com/environments/12415606-e03e06c1-00f4-422a-891a-4e6984aa7cf3?apikey=' + key,
		reporters: 'htmlextra', // htmlextra is a reporting package for newman that will require installlation using NPM.
	    reporter: 
		{ //Go here for documentation: https://www.npmjs.com/package/newman-reporter-htmlextra
			htmlextra: //Report options
			{
				export: '.\\Results.html', //Directory and name
				template: '.\\InnovaAPITemplate.hbs', //Template used for report. Html, CSS and JS are all embedded.
				title: 'Innovations API',
				showEnviornmentData: true,
				browserTitle: "Innovations API",
				showMarkdownLinks: true,                
				showFolderDescription: true,
                //skipSensitiveData: true //Hides request header because it can contain the authorization tokens.               
			}
		}
	},// <-- End of options object
	function (err,summary)// <-- Beginning of callback function
	{
		if (err)
		{ 
			console.log(err);
			throw err; 
        } else
        {
			console.log("summary.environment is: " + summary.environment);
			console.log("summary.globals is: " + summary.globals);
			console.log(summary);
        }
	}// <-- End of callback	
).on('start', function (err, args) { // <-- start event. 
    	console.log('running a collection...');
}).on('done', function (err, summary) { // <-- done event.
    if (err || summary.error) {
        console.error('\nCollection run encountered an error.');
    }
    else {
        console.log('\nCollection run completed.');
    }
})
;

I’ve verified that the UID used in the script for both the collection and environment are correct.

We got there in the end @robmcec :joy:

1 Like

Yes and I am forever grateful!

:partying_face:

1 Like