Newman Run Action

Newman Run is a command-line Collection Runner for Postman. It can be use to run API tests and report on the outcomes.

Newman

Name

A friendly name for this action (will be displayed in the actions workflow area).

Collection File

The path and filename of the Postman collection file. This is a JSON file which can be exported from Postman.

Working Directory

Where to run the Newman command line. Newman will use this working directory while reading files with relative paths. Defaults to workspace directory. [--working-dir]

Folder

The name of a single folder to run within the Postman collection. [--folder]

Iteration Count

Number of iterations to run. [--iteration-count]

Install Newman CLI on agent using NPM

Tick this to automatically install the Newman CLI on the agent using NPM. A new NPM tab will also be displayed with options for the NPM command line.

Using

When not installing using NPM, the "Using" drop down is populated with any property collector whose namespace matches the pattern defined by the Newman action. The pattern for this action is  ^Newman\..*

The default property collector will search the environment PATH for "newman.exe". If you create a property collector for this action, make sure you select the Path Finder PlugIn type and give it a name that will match the pattern above in blue. 

For more in-depth explanations on property collectors see Property Collectors.

Alternatively, you can select the Custom option from the Using drop down list and specify a path in the resulting input field that will be displayed. Please read Why it's a good idea to use a property collector before using this option.

NPM

Install Newman CLI To

The folder NPM should install the Newman CLI to.

Using NPM

The Using drop down is populated by any property collector properties whose namespace matches the pattern defined by the NPM action. The pattern for this action is  ^NPM\..* . The default property collector searches the environment path for "NPM.exe".  

If you create a property collector for this action, make sure you select the Path Finder PlugIn type and give it a name that will match the pattern above in blue.

For more in-depth explanations on property collectors see Property Collectors.

Alternatively, you can select the Custom option from the Using drop down list and specify a path in the resulting input field that will be displayed. Please read Why it's a good idea to use a property collector before using this option.

Files

Environment File

The path to a JSON file containing Postman environment variables. [--environment]

Globals File

The path to a JSON file containing Postman global variables. [–globals]

Iteration Data File

The path to a JSON or CSV file containing data to use for Postman iterations. [--iteration-data]

Cookie Jar File

The path to a JSON file containing a JSON cookie jar in tough-cookie format. [--cookie-jar]

Variables

Postman Global Variables

Global variables to pass to the Postman collection. One key=value pair per line. [--global-var]

Postman Environment Variables

Environment variables to pass to the Postman collection. One key=value pair per line. [--global-var]

Settings

Stop the runner when a test case fails.

Tick to abort the collection run when a test case fails. [--bail]

Prevent reading of files outside of the working directory

Untick this to allow reading files external to the working directory. [--no-insecure-file-read]

Request Delay 

Delay (in milliseconds) between requests. [--delay-request]

Timeout

Time (in milliseconds) to wait for collection run to complete execution. Leave blank (or 0) for no timeout. [--timeout]

Request Timeout

Time (in milliseconds) to wait for each request. Leave blank (or 0) for no timeout. [--timeout-request]

Script Timeout 

Time (in milliseconds) to wait for scripts to complete execution. Leave blank (or 0) for no timeout. [--timeout-script]

Suppress exit code 

Tick to continue running tests even after a failure, but exit with code equal to 0. [--suppress-exit-code]

Suppress all output 

Tick to run without any output to the log file. [--suppress-exit-code]

Show detailed information of collection run and each request sent 

Tick for verbose output. [–verbose]

SSL

Turn off strict SSL

Tick to disable SSL certificate validation. [--insecure]

SSL Client Certificate List

The path to a JSON file containing client SSL certificate configurations. [--ssl-client-cert-list]

SSL Client Certificate

The path to a PEM client SSL certificate file. [--ssl-client-cert]

SSL Client Key

The path to a JSON file containing client SSL certificate private key. [--ssl-client-key]

SSL Client Passphase

An SSL client certificate passphrase (for protected key). [--ssl-client-passphrase]

SSL Extra CA Certificate

The path to an additionally trusted CA certificate. [--ssl-extra-ca-certs]

Export

Collection Export File

The path to a file to export the final the Postman collection JSON to before completing the run. [--export-collection]

Environment Export File

The path to a file to export the final the Postman environment JSON to before completing the run. [--export-environment]

Globals Export File

The path to a file to export the final the Postman global JSON to before completing the run. [--export-globals]

Cookie Jar Export File

The path to a file to export the final the Postman cookie jar JSON to before completing the run. [--export-cookie-jar]

Reporters

Reporters

Optionally select one or more reporters for outputting the test results to. The JUnit XML reporter is required to display results on the Unit Tests tabs of the Build page. The output from HTML reporter can optionally be automatically registered as a report. Note that the CLI reporter will be added automatically so that the you can view results in the build log.  [--reporter]

Use HTML Extra reporter

Shown when the HTML reporter option is selected. Tick to use the HTML Extra reporter to produce a richer HTML report. Note that the HTML Extra reporter is not built-in to newman and must be installed.

Use JUnit Full reporter

Shown when the JUnit XML reporter option is selected. Tick to use the  JUnit Full reporter to produce a richer Junit report. Note that the JUnit Full reporter is not built-in to newman and must be installed.

Custom Reporter

The name of a custom reporter to use.

Additional Reporter Arguments

Additional arguments for reporters. One argument value pair per line.

Install Newman HTML Extra Reporter on agent using NPM

Tick to automatically install the Newman HTML Extra Reporter using the same NPM settings used to install Newman

Install Newman JUnit Full Reporter on agent using NPM

Tick to automatically install the Newman JUnit Full Reporter using the same NPM settings used to install Newman

Register all tests as unit tests

Shown only when the JUnit reporter is selected. Tick to register test results with Continua so that that are displayed on the Unit Test tab of the Build page.

Automatically register HTML report with the server

Shown only when the HTML reporter is selected. Tick to automatically create the necessary workspace rules and report registration to add the HTML report to the build Reports tab. Note that the HTML report export file needs to be in the workspace to register the report with the server.

HTML Reporter Export Path

Shown only when the HTML reporter is selected. The path to export the HTML report to. Defaults to a unique file in the 'newman-reports' subfolder of working folder. [--reporter-html-export | --reporter-htmlextra-export]

HTML Reporter Template Path

Shown only when the HTML reporter is selected. The path to a custom Handlebars template used to render the HTML report. [--reporter-html-template | --reporter-htmlextra-template]

JUnit Reporter Export Path

Shown only when the JUnit reporter is selected. The path to export the JUnit report to. [--reporter-junit-export  | --reporter-junitfull-export] 

JSON Reporter Export Path

Shown only when the JSON reporter is selected. The path to export the JSON report to. [--reporter-json-export].

Report

Name

The name of the report - shown on the report selection tab. This defaults to a name derived from the name of the action.

Description

A description for the report - shown as the tooltip title on the report selection tab.

Run Order

The order in which the reports are displayed. Reports with higher run order numbers appear to the right of those with lower numbers. If run order numbers are equal then reports are ordered by report creation time.

Options

Fail action if any tests fail

Tick this to cause the build to fail if any tests fail.

Fail action if any tests error

Tick this to cause the build to fail if an error occurred while running any test.

Fail action if any tests are inconclusive

Tick this to cause the build to fail if any tests are inconclusive.

Timeout (in seconds)

How long to wait for the action to finish running before timing out. Leaving this blank (or zero) will default to 86400 seconds (24 hours).

Treat failure as warning

Tick to continue build on failure marking the action with a warning status.

Ignore warnings

If this is ticked, any warnings logged will not mark the action with a warning status.

Environment

Environment Variables

Multiple environment variables can be defined - one per line. These are set before the command line is run.

Log environment variables

If this is ticked, environment variable values are written to the build log. 

Generate system environment variables

Tick this checkbox to set up a list of new environment variables prefixed with 'ContinuaCI.' for all current system expression objects and variables.

Mask sensitive variable values in system environment variables

This checkbox is visible only if the 'Generate system environment variables' checkbox is ticked.

If this is ticked, the values of any variables marked as sensitive will be masked with **** when setting system environment variables. Clear this to expose the values.