Collection runner

The collection runner sends every request in a collection or folder in order, runs their pre-request and post-response scripts, and records each test result. Use it to check that an API still works end to end, to replay a sequence of calls with different data, or to produce a test report. It works like Postman’s Collection Runner and Newman, and the same runs are available from the command line with cloakctl collection run.

12345

The runner: options and requests on the left, results as they arrive on the right.

1. Iterations, delay, stop on first failure and keep variable changes
2. Choose a dataset or a CSV/JSON data file
3. Uncheck a request to leave it out of the run
4. Passed and failed tests, request count, duration and average response time
5. Export the results as JSON, JUnit XML or an HTML report

×

Open the runner

The runner opens in its own tab, titled Run · Collection name. Open it from any of these places:

Where	What to click
Collection tab	Run in the header
Sidebar, collection row	Right-click › Run Collection
Sidebar, folder row	Right-click › Run Folder
Folder tab	Run Folder in the header
Dataset tab	Run With… › a collection (the dataset is selected for you)
Global search	Type the collection name and choose Run Collection name (Collection Runner)

Run Folder runs only the requests inside that folder and its subfolders. Each collection and folder keeps its own runner settings and last results while the app is open, so you can switch tabs and come back.

1

Run, Load Test and Mock are in every collection's header.

1. Run opens the collection runner

×

Run a collection

1. Open the runner for the collection or folder.

2. Set Iterations to the number of times to run the requests. Leave it at 1 for a single pass.

3. Optionally choose data under Data: none › Choose: a dataset or a CSV or JSON file. Each row becomes one iteration. See Datasets and data files.

4. Under Requests, uncheck any requests you don’t want to send.

5. Click Run Collection name, or press ⌘ ↩.

   Results appear on the right as each request finishes. While the run is going, the header shows the current iteration and how many requests have finished. Click Stop to cancel.

The runner uses the active environment from the environment picker, your globals, the collection’s variables and your Vault secrets.

Run options

Option	Default	What it does
Iterations	1	How many times to run the selected requests, from 1 to 10,000. Disabled while a dataset or data file is chosen, because the number of rows sets the iteration count.
Delay	0 ms	Waits this long between requests in an iteration, from 0 to 60,000 ms in steps of 100. There’s no wait after the last request of an iteration.
Stop on first failure	Off	Ends the whole run at the first request that errors or has a failed test.
Keep variable changes	On	After the run, saves the changes scripts made to globals, the active environment and the collection’s variables (pm.*.set updates or adds a variable, pm.*.unset removes it). Turn it off to run against a copy and leave your variables as they were.
Data	none	A dataset or a CSV or JSON data file. One iteration per row; columns are available as {{column}} and pm.iterationData. Remove clears it.

The Choose menu lists every dataset with its row count, then Data File (CSV or JSON)… to pick a file, and New Dataset to create one. A dataset is read when the run starts, so edits you make to it apply to the next run.

Choose which requests run

The Requests list shows every request the runner can send, in collection order, with its folder path (for example, Other protocols / Countries (GraphQL)). Each one has a checkbox:

• Checked requests run in the order shown.
• Unchecked requests are skipped, and pm.execution.setNextRequest() can’t jump to them.

Your selection is kept for the collection while the app is open.

Which request types run

The runner sends HTTP and GraphQL requests, plus these protocols:

Protocol	What the runner does	What the response body holds
gRPC	Calls the saved method. For client and bidirectional streaming, a JSON array message sends one message per item.	The response message, or an array of messages for streaming calls. The gRPC status maps to an HTTP status, with grpc-status and grpc-message headers.
MCP	Calls the saved tool, or lists the tools when no tool is chosen.	The tool result. A tool that returns an error answers 500.
AI	Sends the saved prompt to the chosen provider and model.	The model’s text, with token usage in x-ai-* headers.
SQL and data	Runs the saved query.	The rows of the last result set, as JSON objects.
WebSocket, Socket.IO, MQTT	Connects, sends the composed message, and collects replies until the connection has been quiet for half a second.	The message received, or an array of messages.
TCP and UDP	Same as WebSocket, for requests in Client mode. Server and Forward & Record requests aren’t listed.	The data received.

Each protocol answers with an HTTP-style response, so the same pm.response tests work everywhere. For example, a gRPC NOT_FOUND becomes a 404 with a grpc-status header, and a failed MCP tool becomes a 500.

How a run works

Within each iteration the runner sends the checked requests from top to bottom. For each request it runs the collection, folder and request pre-request scripts, sends the request, then runs the post-response scripts and tests in the same order. See Scripts and tests.

Change the order from a script

Scripts can steer the run:

Call	Effect
pm.execution.setNextRequest("Get pet")	Runs the request with that name (or ID) next. Unknown names are ignored and the run continues in order. Also available as postman.setNextRequest().
pm.execution.setNextRequest(null)	Ends the current iteration. The next iteration starts from the top.
pm.execution.skipRequest()	In a pre-request script, skips sending this request. It shows as skipped in the results.

Post-response script: poll until a job is done

const job = pm.response.json();
if (job.state !== "done") {
  pm.execution.setNextRequest("Get job status");
}

Loops have a limit

setNextRequest can loop, but not forever. If an iteration sends more than 100 times the number of requests plus 1,000, the run stops early.

Variables across iterations

• Each iteration gets the next row of data (if any) as the data scope, which overrides environment, collection and global values. Local values set by a script still win.
• Changes scripts make to globals, environment and collection variables carry over to later requests and later iterations in the same run.
• pm.info.iteration is the current iteration (starting at 0) and pm.info.iterationCount is the total.

Read the results

The results header shows passed and failed test counts, then the number of requests, the total duration and the average response time. Each row in the list shows:

Part	Meaning
#1, #2, …	The iteration the request ran in.
Method and name	The request’s method and its folder path.
Status	The response status code: green below 400, red at 400 and above. skipped when a script skipped the request, error when it couldn’t be sent.
Time	How long the request took.
URL	The URL as sent, with variables filled in.
Error	Why the request failed to send (for example, a connection error), in red.
Tests	One line per test: a check mark for passed, a cross with the error message for failed, a dashed circle for skipped.

A run passes when no test fails and every request was sent. A 404 or 500 response is not a failure by itself. Add a test, such as pm.response.to.have.status(200), when a status code matters.

Tip

Results appear in the order requests ran, so a setNextRequest jump or a skipped request is easy to follow.

Export the results

When a run finishes, choose Export Results and a format. Each one is saved where you choose.

Menu item	File	Use it for
JSON…	run.json	Your own tooling and dashboards.
JUnit XML…	run.junit.xml	CI systems that read JUnit reports.
HTML Report…	run.html	A readable report to share. It opens in your browser after saving.

JSON

run.json

{
  "collection": "Petstore",
  "startedAt": "2026-10-13T09:07:20Z",
  "durationMs": 97,
  "iterations": 1,
  "totals": {
    "requests": 1,
    "failedRequests": 0,
    "tests": 2,
    "passed": 2,
    "failed": 0,
    "skipped": 0
  },
  "executions": [
    {
      "iteration": 0,
      "path": ["List pets"],
      "method": "GET",
      "url": "https://api.example.com/v1/pets",
      "responseTimeMs": 84,
      "tests": [
        { "name": "Status code is 2xx", "passed": true, "skipped": false, "error": null },
        { "name": "Responds within 500 ms", "passed": true, "skipped": false, "error": null }
      ],
      "status": 200,
      "responseSize": 512
    }
  ]
}

status and responseSize appear only when a response arrived; error appears when the request couldn’t be sent.

JUnit XML

run.junit.xml

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Petstore" tests="2" failures="0" errors="0" time="0.097">
  <testsuite name="List pets" tests="2" failures="0" errors="0" time="0.084" timestamp="2026-10-13T09:07:20Z">
    <testcase name="Status code is 2xx" classname="List pets" time="0.084"></testcase>
    <testcase name="Responds within 500 ms" classname="List pets" time="0.084"></testcase>
  </testsuite>
</testsuites>

There’s one testsuite per request sent (named by its folder path) and one testcase per test. A failed test gets a failure element of type AssertionFailure; a skipped test gets skipped; a request that couldn’t be sent gets an extra testcase with an error element.

HTML

The HTML report is a single file with no external resources. It shows the collection name, start time, iteration count and duration; cards for requests, passed, failed and average response time; and a table with #, Method, Request (with the URL), Status, Time and Tests. Rows with a failure have a red edge. It follows the reader’s light or dark mode.

Scripts, tests and packages in a run

Everything that works when you send one request works in the runner: collection, folder and request scripts, pm.test and pm.expect, variables, pm.sendRequest, cookies and Vault secrets. Shared code from the Packages section loads with pm.require("name") in the app and in cloakctl.

1

Packages are CommonJS modules shared by every script, including scripts that run in the runner.

1. Copy Import copies the pm.require line for this package

×

See Scripts and tests for the full pm API and packages.

Run from the command line

cloakctl collection run runs a collection without the app, like Newman or the Postman CLI. It reads the same workspace folder as the app, so it sees your collections, environments, datasets and packages.

Terminal

cloakctl collection run "Petstore" -e Staging -d "Pet logins" -r cli,junit --reporter-junit-export results.xml

Argument or flag	What it does
NAME or FILE.json	A collection in the workspace (by name or ID), or a Postman collection file or OpenAPI spec file.
-e, --environment ENV	An environment by name, or a Postman environment file. Defaults to the workspace’s active environment.
-g, --globals FILE.json	A Postman globals or environment file to use instead of the workspace globals.
-d, --iteration-data DATA	A CSV or JSON data file, or the name of a dataset in the workspace. One iteration per row.
-n, --iteration-count N	Number of iterations. Default 1. Ignored when data is given.
--folder NAME	Run only this folder (searched by exact name, including nested folders).
--delay-request MS	Wait between requests.
--bail	Stop at the first failed test or request error.
--timeout-request MS	Set the timeout of every request.
--insecure, -k	Turn off TLS certificate verification for every request.
--env-var KEY=VALUE	Set an environment variable for this run. Repeat for more. Overrides the environment’s value.
-r, --reporters LIST	Comma-separated: cli, json, junit, html. Default cli.
--reporter-json-export FILE	Where to write the JSON report. Default cloak-run.json.
--reporter-junit-export FILE	Where to write the JUnit report. Default cloak-run.xml.
--reporter-html-export FILE	Where to write the HTML report. Default cloak-run.html.
--workspace DIR	Use a different workspace folder.

The cli reporter prints each request as it finishes, with its tests, console.log output and script errors, then a summary:

Output

Petstore

→ List pets
  GET https://api.example.com/v1/pets [200 OK, 512 bytes, 84ms]
  ✓ Status code is 2xx
  ✓ Responds within 500 ms

requests 1 (0 failed) · tests 2 (0 failed) · avg response 84ms · total 97ms

The exit code is 0 when every test passes and every request was sent, and 1 otherwise, so CI fails the build on a broken test. Variable changes from scripts aren’t written back to the workspace. See Headless and CI for pipeline examples and cloakctl for every command.

Use --insecure with care

--insecure accepts any certificate, including one presented by an attacker. Use it only against your own test servers with self-signed certificates.

Related

Datasets and data filesRun once per row of test data.

Scripts and testsThe pm API, assertions and packages.

Monitors and webhooksRun a collection on a schedule.

Run API tests in CIUse cloakctl and JUnit reports in a pipeline.