Unable to run scripts from Package Library via Newman

hi,

I created a new collection that uses packages instead of using eval=>…

Locally in Postman, it’s working just fine but when I tried to run it via Newman all the scripts were failing.

This is my Job: I am triggering collection sanity_RF this is the only collection that use packages

name: Test API collection

on:
  workflow_dispatch:
    inputs:
      Projects:
        description: 'Projects'
        required: true
        default: ''
        type: choice
        options:
          - "Select a project"
          - All Booking tests
          - Sanity dev integration
          - Sanity_RF
          - Price_calculation_baggage
          - Schema_BELLTRAV_via_12G0
          - Schema_NOKNARAR_via_12G0
          - Sanity prod
          - Search
          - Compare_Search_TC_vs_BAW
      environments:
        description: 'Environments'
        required: true
        default: 'integration'
        type: choice
        options:
          - integration
          - dev
          - production
          - production_bookaway

jobs:
  newman:
    name: "Collection: ${{ inputs.Projects }}"
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

  run-tests:
    runs-on: ubuntu-latest
    name: "Environment: ${{ inputs.environments }}"

    steps:
      - name: Clear cache
        run: |
          rm -rf ~/.npm
          rm -rf ~/.cache

      - uses: actions/checkout@v4
        with:
          ref: ${{ github.ref }}

      - name: Install Node
        uses: actions/setup-node@v4
        with:
          node-version: "20.x"

      - run: npm i npm@latest
      - run: npm i

      - name: Install newman
        run: |
          npm install -g newman
          npm install -g newman-reporter-htmlextra

      - name: Run POSTMAN collection
        id: newman
        run: |
          if [ -n "${{ inputs.environments }}" ]; then
            envFile="${{ inputs.environments }}"
            case $envFile in
              integration)
                envFilePath="integration.postman_environment.json"
                ;;
              dev)
                envFilePath="dev.postman_environment.json"
                ;;
              production)
                envFilePath="production.postman_environment.json"
                ;;
              production_bookaway)
                envFilePath="production_bookaway.postman_environment.json"
                ;;
              *)
                echo "Invalid environment selection: $envFile"
                exit 1
                ;;
            esac
          else
            envFilePath="integration.postman_environment.json"
          fi

          # Handle project selection
          project="${{ inputs.Projects }}"
          case $project in
            "Sanity dev integration")
              collections=("./Sanity/Sanity.postman_collection.json")
              ;;
            "Sanity_RF")
              collections=("./Sanity/Sanity_RF.postman_collection.json")
              ;;
            "Sanity prod")
              collections=("./Sanity_denaly/Sanity_Daneli_proxy.postman_collection.json")
              ;;
            "Search")
              collections=("./Search/Search.postman_collection.json")
              ;;
            "Price_calculation_baggage")
              collections=("./Price_calculation_baggage/Price_calculation_NOKNARAR_via_12G0.postman_collection.json")
              ;;
            "Schema_BELLTRAV_via_12G0")
              collections=("./Schema_BELLTRAV_via_12G0/Schema_BELLTRAV_via_12G0.postman_collection.json")
              ;;
            "Schema_NOKNARAR_via_12G0")
              collections=("./Schema_NOKNARAR_via_12G0/Schema_NOKNARAR_via_12G0.postman_collection.json")
              ;;
            "Compare_Search_TC_vs_BAW")
              collections=("./Compare_Search_TC_vs_BAW/Compare_Search_TC_vs_BAW.postman_collection.json")
              ;;
            "All Booking tests")
              if [ "$envFile" == "integration" ]; then
                collections=(
                  "./Sanity/Sanity.postman_collection.json"
                  "./Schema_BELLTRAV_via_12G0/Schema_BELLTRAV_via_12G0.postman_collection.json"
                  "./Schema_NOKNARAR_via_12G0/Schema_NOKNARAR_via_12G0.postman_collection.json"
                )
              else
                collections=(
                  "./Sanity/Sanity.postman_collection.json"
                  "./Price_calculation_baggage/Price_calculation_NOKNARAR_via_12G0.postman_collection.json"
                  "./Schema_BELLTRAV_via_12G0/Schema_BELLTRAV_via_12G0.postman_collection.json"
                  "./Schema_NOKNARAR_via_12G0/Schema_NOKNARAR_via_12G0.postman_collection.json"
                )
              fi
              ;;
            "Select a project")
              echo "Please select a valid project."
              exit 1
              ;;
            *)
              echo "No valid project selected for $project. Skipping..."
              exit 0
              ;;
          esac

          for collectionPath in "${collections[@]}"; do
            if [ -n "$collectionPath" ]; then
              # Run the collection
              newman run "$collectionPath" -e "./Environments/$envFilePath" --reporters cli,htmlextra --reporter-htmlextra-export "docs/index-$(basename "$collectionPath" .postman_collection.json).html"
              status=$?  # Save the exit code in a variable
              echo "Newman exit code for $(basename "$collectionPath" .postman_collection.json): $status"
            fi
          done

      - name: Upload HTML reports as artifacts
        if: ${{ success() || failure() }}  # Upload artifact regardless of previous step outcome
        uses: actions/upload-artifact@v4
        with:
          name: HTML-Reports
          path: docs/*.html

      # Post to a Slack channel if the Newman run fails
      - name: Post to a Slack channel
        if: ${{ failure() }}  # Only run if the Newman run fails
        uses: slackapi/slack-github-action@v1.25.0
        with:
          channel-id: 'C06Q6007SMT'
          slack-message: |
            GitHub build result: *Failed* ❌ 
            *Collection: ${{ inputs.Projects }}*
            *Environment: ${{ inputs.environments }}*
            ${{ github.event.pull_request.html_url || github.event.head_commit.url }} 
            ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} 
            Build triggered for branch: *main*
        env:
          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}

      # Post to a Slack channel if the Newman run passes
      - name: Post to a Slack channel
        if: ${{ success() }}  # Only run if the Newman run passes
        uses: slackapi/slack-github-action@v1.25.0
        with:
          channel-id: 'C06Q6007SMT'
          slack-message: |
            GitHub build result: *Passed* âś… 
            *Collection: ${{ inputs.Projects }}*
            *Environment ${{ inputs.environments }}*
            ${{ github.event.pull_request.html_url || github.event.head_commit.url }}
            ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
            Build triggered for branch: *main*
        env:
          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}

Hey @technical-engineer16 :wave:

The Package Library is not available in Newman and only runs with the Postman CLI, for certain plan types.

  1. WHY ITS NOT MENTIONED IN DOCUMENTATION Reuse scripts in Postman | Postman Learning Center
  2. SO HOW CAN I USE IT IN GIT ACTION JOB? PLEASE PROVIDE AN EXAMPLE

I’m very sure that using all caps wasn’t really required for your reply.

That learning center page explains how the packages can be created and used within the Postman platform.

Newman is an independent npm package which isn’t the Postman UI.

How would your static exported Collections be aware of the packages you’re using in the UI?

I didn’t get this question (How would your static exported Collections be aware of the packages you’re using in the UI?).

I have used eval for reuse of function and it worked fine, but if I am PAYING to Postman I should be able to continue to do the same but with packages.

This is how I am implementing it with newman:

for collectionPath in "${collections[@]}"; do
            if [ -n "$collectionPath" ]; then
              # Run the collection
              newman run "$collectionPath" -e "./Environments/$envFilePath" --reporters cli,htmlextra --reporter-htmlextra-export "docs/index-$(basename "$collectionPath" .postman_collection.json).html"
              status=$?  # Save the exit code in a variable
              echo "Newman exit code for $(basename "$collectionPath" .postman_collection.json): $status"
            fi
          done

***So my question is can packages work in Git action with CLI postman? ***
If yes can you provide a short example of how it should be done?

Your exported JSON file just happens to be in the Collection format that is valid when run with Newman. It has no direct connection with anything in the Postman UI - When you run that, do you see any evidence of that being run in the Postman UI? The Postman platform has no way of knowing that you run a Collection with Newman in that way.

The Postman CLI has a direct connection and uses your Postman API Key to login. When running the Collection with the CLI using the id, it appears in the runs section of the Collection and you can see the run summary.

Newman and the Postman CLI are different tools and both equally do things in different ways, they don’t have a 1:1 feature parity.

Using eval and with code that placed in the Collection, Global file or Environment file will work because it’s exported in those artifacts - The Package Library and the scripts within it are not exported in those files.

Yes, Package Library will work with the Postman CLI but only for certain licence types.

A basic example of the GH Actions syntax for your Collections can be extracted from the Runner by selecting the Configure command link

name: Automated API tests using Postman CLI

on: push

jobs:
  automated-api-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install Postman CLI
        run: |
          curl -o- "https://dl-cli.pstmn.io/install/linux64.sh" | sh
      - name: Login to Postman CLI
        run: postman login --with-api-key ${{ secrets.POSTMAN_API_KEY }}
      - name: Run API tests
        run: |
          postman collection run "<collection-id>" -e "<environment-id>"

Thanks now it’s more clear.

1 Like

Hi,
after reviewing this amazing technology I have one more question.

According to the pricing page:

There is a limit on “Collection Runs” so it means if I use “Postman CLI” in Git action I will be limited to running 25 collections per month???

IF yes - please provide a solution. We run daily jobs each hour and more.

else

Explain the Collection Runs"

Collection Runs are the runs made from the Collection Runner inside the platform.

The Postman CLI usage would be based on the Postman API limit, which for a free user is 1000 calls a month.

If you’re using the Postman CLI with a static JSON there wouldn’t be a limit attached to that, like if you’re using Newman. If you use it like this, you wouldn’t get the additional benefits of tracking the runs and the reports from them in your Collections and you also wouldn’t have access to the Package Library.

Hi, I need access to the packages what will be the way to use the Postman CLI to access the packages?
My projects are based on reusable functions.

If you’re using the Postman CLI by logging in using your Postman API Key and referencing the Workspace elements by id - Then it will pick up those scripts from the package library.

This is only available for certain licences, as I’ve mentioned a couple of times now in this thread.

I have a professional licence will it work?
Also i see that newman HTML report is not longer available is there any alternative for reporting?

Have you tried it? Are you seeing a specific error when using it?

Currently, the custom reporting is not part of the Postman CLI.

hi , i was able to use packages and the job is working fine with the secrets.POSTMAN_API_KEY, just one thing is missing .
It will be great to get a report, for now all i see if its failing


but there is no log info why, in newman it was clear.
So nothing for the alternative?
If there is nothing how can i get the the log into a file?

For that Collection, if you open that in a tab in the UI, you will see a Runs tab which will show that run and a basic report.

Currently, as I mentioned, custom reporting is not available. This is an area that will be iterated on and new functionality will be added to the Postman CLI.

This is only available for certain licences

Could you please specify which license is required for packages to work with postman CLI?

We are currently using a free plan and I have hit this limitation. I am investigating if Postman is the right tool for us.

Collection Runs are the runs made from the Collection Runner inside the platform.

Could you elaborate what does “inside the platform” mean?
Do Postman UI manual runs count?
Do runs from monitors count?
Do scheduled runs count?

I am not sure if scheduled runs is the same thing as a monitors (I am using monitors), is it the same thing?

Thank you

I’ve upgraded our plan to a paid Basic plan.

Still receiving the error “Packages are not supported in Postman CLI for your plan.”.

Which plan enables the packages?

Found the answer: Executing packages from the Postman CLI is available on Postman Professional and Enterprise plans

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.