Newman not generating HTML report on GitHub CI/CD pipeline

Ask the Experts and Postman Tips
, , , ,
1

Your question may already have an answer on the community forum. Please search for related topics, and then read through the guidelines before creating a new topic.

Here’s an outline with best practices for making your inquiry.

My question:
I’ve recently started using newman to execute API tests since Postman CLI doesn’t generate an HTML report. While testing locally using the ‘newman run’ command, I’m able to generate the HTML report. However, when running the command on GitHub CI/CD, the file isn’t being generated and I’m not receiving much error information other than:

“Run newman run ‘Postman Collections/APICollection.json’ --globals ‘Postman Collections/workspace.postman_globals.json’ -r htmlextra --reporter-htmlextra-export htmlreport.html
newman run ‘Postman Collections/APICollection.json’ --globals ‘Postman Collections/workspace.postman_globals.json’ -r htmlextra --reporter-htmlextra-export htmlreport.html
shell: /usr/bin/bash -e {0}
Error: Process completed with exit code 1.”

I’m uncertain as to what the issue is. The API tests appear to be running correctly, but the report generation isn’t working. Can you please confirm if the newman command I’m using is correct? I’ve provided the GitHub path below.

Details (like screenshots):

Github Repo Structure

r-postman-collection/
├── .github (Folder)
    ── workflows (under .github) 
        ── schedule.yml (under workflows) 
├── Postman Collections (Folder)
    ── APICollection.json (under Postman Collections) 
    ── workspace.postman_globals.json (under Postman Collections) 

YML File

name: Newman API Tests
on: 
  workflow_dispatch:
jobs: 
  test: 
    defaults: 
      run: 
        working-directory: ./tests
    environment: staging
    name: "Run tests"
    runs-on: ubuntu-latest
    steps: 
      - 
        uses: actions/checkout@v2
      - 
        uses: actions/setup-node@v2
        with: 
          node-version: "18"
      - 
        name: "Install newman"
        run: "npm install -g newman"
      - 
        name: "Install newman reporter"
        run: "npm install -g newman-reporter-htmlextra"
      - 
        name: "Run tests"
        run: "newman run NewmanDemo.postman_collection.json -e ReqResDev.postman_environment.json  
        -r htmlextra"
      
      - name: Publish report
        uses: MeilCli/slack-upload-file@v1
        with:
          slack_token: ${{ secrets.SLACK_TOKEN }}
          file_path: './tests/htmlreport.html'
          file_name: 'postmanReport.html'
          file_type: 'html'
          channels: 'api-test-reports'
2

@danny-dainton Would you be able to help me out here?

Wondering, do I need to allow any permissions from GitHub to allow Newman to generate HTML file under repo?

3

Not going to lie, I don’t know how it works with GH actions. I’m familiar with the way that you store the report artifacts in Gitlab for pipeline run though. :grimacing:

I found this is the GH docs, I don’t know if there is a certain GH actions command to use in your yml file to tell it to store the artifacts :thinking:

4

What worked for me here is to upload the artifact generated from the GHA newman execution to the container which makes it available for download etc. I also piped the console log into a file and did the same thing.

–reporters cli,htmlextra --reporter-htmlextra-export {{nameForReport}}.html | tee newman.log

- name: Newman Log Output File Storage
  uses: actions/upload-artifact@v3  # store log file of the newman output
  if: success() || failure()        # run this step even if previous step failed
  with:
    name: Console-Log
    path: newman.log

- name: HTML Extra File Storage
  uses: actions/upload-artifact@v3  # store artifact containing html report
  if: success() || failure()        # run this step even if previous step failed
  with:
    name: Test-Report
    path: {{nameForReport}}.html
1 Like