# Release diff

> Record the same journey on two builds, or yesterday and today, and Cloak Proxy compares them endpoint by endpoint — endpoints added and no longer called, fields added, removed or retyped, status changes, slower calls, extra requests and new third parties receiving user data — with the requests on both sides, MCP and a CI exit code.

Source: https://docs.subcloak.com/cloak-proxy/testing/release-diff/

Added in Cloak Proxy 0.4.0.

A release changes more on the network than its changelog says. A field becomes a string, a call fires three times instead of once, a new SDK starts sending the user's email to an analytics company. **Release Diff** answers "what did this release change on the network?": record the same journey on the old build and the new one, and it compares the two recordings endpoint by endpoint.

[Schema changes](https://docs.subcloak.com/cloak-proxy/insights/schema-changes/) spot a response that changes shape within one session. Release Diff compares whole sessions, so it also sees endpoints that stopped being called, requests that got slower or more frequent, and data that started going to someone new.

*[Screenshot: The Release Diff window comparing ShopApp 2.3 with ShopApp 2.4: 19 differences, 6 high, 7 medium and 6 low. The Before and After cards show each side's API requests and endpoints with a Change button. The sidebar lists All differences, Endpoints, Fields, Status codes, Speed, Request counts and Third parties with counts, then Every endpoint and Every third party. The list shows, most severe first, Type changed at $.items[*].price for GET /v2/products, Type changed at $.price, Field removed at $.loyalty for GET /v2/users/me, Status changed for POST /v2/orders (now returns 500 in 1 of 2 calls), New third party AppsFlyer, Sends new data to Mixpanel, Endpoint no longer called for GET /v2/recommendations, Now null at $.trackingUrl, Query parameter removed ?limit, Slower and More requests for GET /v2/products/{id}. The selected type change's detail pane shows number before and string after, request #12004 on 2.3 and #12020 on 2.4, and the endpoint's calls, median and statuses.. Every difference between two builds, most severe first. The detail pane shows the values on each side and the requests behind them.]*

## Compare two builds

1. Record the journey on the old build: sign in, browse, add to the cart, check out — whatever the release touches.

2. Open **Tools › Release Diff…** (<kbd>⇧</kbd> <kbd>⌥</kbd> <kbd>⌘</kbd> <kbd>D</kbd>) and choose **Keep Current Traffic as Before**. Cloak Proxy saves a copy of the session for the comparison, so you can clear the traffic list.

3. Clear the session, install the new build and record the same journey.

4. Choose **After › Change › All Captured Traffic** (or one app, host or version) and click **Compare** (<kbd>⌘</kbd> <kbd>↩</kbd>).

> **On Linux:**
> Release Diff opens with <kbd>Ctrl</kbd> <kbd>Alt</kbd> <kbd>Shift</kbd> <kbd>D</kbd>, and **Compare** is <kbd>Ctrl</kbd> <kbd>Enter</kbd>. **Session File…** opens the desktop's file chooser, and exports go through its Save dialog.

You don't have to clear the session in between. When the app puts its version in its User-Agent (`ShopApp/2.4 (iPhone; iOS 26.0)`, which most iOS and Android HTTP stacks do), or uses the [in-app SDK](https://docs.subcloak.com/cloak-proxy/devices/in-app-sdk/), both builds can be recorded into one session: each side's **Change** menu lists the **App Versions** it found. Right-click an app in the sidebar and choose **Release Diff for** the app to compare its two newest versions straight away.

### Where each side comes from

Each side's **Change** menu offers:

| Source | What it compares |
|---|---|
| **All Captured Traffic** | Everything in the traffic list |
| **App Versions** | One app at one version, such as **ShopApp 2.4 (52)** (the number is its requests) |
| **App** / **Host** | One app's or one host's requests |
| **Selected Requests** | The requests selected in the traffic list. Right-click a selection and choose **Diff Against Another Recording…** to make it the after side |
| **Session File…** | A saved `.cloaksession` or `.har` file, such as last week's recording or one from a teammate |
| **Keep Current Traffic as Before** | A copy of the traffic list as it is now (before side only) |

The cards show what each side holds once compared: its API requests, endpoints, and where it came from. **⇄** swaps the sides. When you change a side or an option after comparing, a bar offers **Compare Again**.

## What's compared

Requests are grouped into endpoints the way [Endpoints](https://docs.subcloak.com/cloak-proxy/insights/endpoints/) does — `GET /v2/orders/42` and `GET /v2/orders/57` are both `GET /v2/orders/{id}` — and each endpoint is compared across the two sides. Images, scripts, styles, fonts, tunnels and requests still in flight are left out.

| Difference | When it's reported | Severity |
|---|---|---|
| **New endpoint** | Called after, never before | Low |
| **Endpoint no longer called** | Called before, never after. If the journey was the same, the screen that used it may have changed or broken | Medium |
| **Field removed** | A JSON field in successful responses before is in none after | High if it was in every response before; medium if it was optional |
| **Field added** | A field in responses after that wasn't there before | Low |
| **Type changed** | A field was a number and is now a string, say (`number → string`) | High in a response; medium in a request body |
| **Now null** | A field that was never `null` is `null` in some responses after | Medium |
| **Query parameter removed** / **added** | The app stopped sending `?limit`, or started sending `?sort` | Medium / low |
| **Status changed** | A status the endpoint didn't return before: a 5xx or no response is high, a 4xx or a different success code is medium; an error that went away is low | High, medium or low |
| **Content type changed** | Successful responses were `application/json` and are now `text/csv` | Medium |
| **Slower** | The median time is at least 1.5× the old one and at least 100 ms slower (both in **Options**) | Medium; high when over 3× and a second slower |
| **More requests** | An endpoint is called at least 25% more often and at least once more — a duplicated call, a retry or a new refresh | Medium; high when over 3× and three or more extra |
| **More requests** (whole journey) | The journey made at least 5 and 20% more API requests | Medium |
| **New third party** | A company receives user data after and received none before | High for personal data (email, phone, name, location, device, advertising and user IDs, your app's tokens); medium for IDs from your backend; low when it's contacted with no user data |
| **Sends new data** | A company already receiving data now gets a new kind, such as the email address | High or medium, as above |
| **No longer sent data** | A company stopped receiving user data | Low |

Fields are compared across **every successful JSON response** of the endpoint on each side (up to 60), request bodies across every JSON request body, and both are read after decoding gzip, deflate and Brotli. A field's path is a JSONPath with `[*]` for array items (`$.items[*].price`); when a whole object is added or removed, only the object is listed, not each field inside it. Things that aren't changes are left out: the same fields with different values or in a different order, an array that's empty on one side, maps keyed by IDs (`{"8812": …}`), and cache-busting query parameters (`_`, `t`, `ts`, `cb`, `nonce`…).

Third parties come from the [Privacy Map](https://docs.subcloak.com/cloak-proxy/insights/privacy-map/)'s scanner run on each side, so the same data kinds and company names apply. The app's own domains are guessed from its name and its busiest domain; set them in **Options › App's Own Domains…** if the guess is wrong.

## Read the results

The sidebar groups the differences: **All differences**, then **Endpoints**, **Fields**, **Status codes**, **Speed**, **Request counts** and **Third parties**, each with a count and its most severe icon. **Hide low** leaves out additions and the other changes that rarely break anything.

Each row says what changed (with the field path in the severity's colour), one sentence about it, where it is — the endpoint, or the company and its domain — and, when there's room, the value before and after. Select a row for its detail:

- the sentence in full, with how sure it is: "in all 3 samples before, in none of 2 after", or "it was optional before, so this may be sampling";
- the **before** and **after** values;
- the **requests** on each side that show the difference — click one to compare it with the first request on the other side, or **Show in Traffic** to select them in the traffic list (for a side from the captured traffic);
- the endpoint's calls, median time and status codes on both sides, and its other changes.

**Compare** opens the two requests side by side, line by line, with JSON pretty-printed and keys sorted so formatting doesn't count, and the changed field in bold.

*[Screenshot: Two GET /v2/products responses side by side: ShopApp 2.3's request #12004 on the left and ShopApp 2.4's #12020 on the right. Lines with price are highlighted, 89 in red on the left and the string 89.00 in green on the right, and the same for 12.50 and 129.00. A Request and Response switch, an Only changes checkbox and 3 lines differ at the top.. Compare: the before and after requests line by line. Here the price became a string.]*

### Every endpoint

**Every endpoint** lists each endpoint on either side, changed ones first: **New** and **Gone** badges, calls before → after, the median time (in red when it's slower), the status codes on each side and how many differences it has. Select one to see its changes and its requests on both sides.

*[Screenshot: Every endpoint: POST /v2/orders called 1 → 2 times, GET /v2/products/{id} called 1 → 3 times with a median of 148 ms → 637 ms in red, GET /v2/recommendations marked Gone, GET /v2/wishlist and the AppsFlyer event endpoint marked New, and unchanged endpoints with an equals sign. The selected GET /v2/products/{id} lists its changes (Type changed at $.price, Slower, More requests) and its one request on 2.3 next to three on 2.4.. Every endpoint side by side: calls, median time, statuses and how many differences each has.]*

### Every third party

**Every third party** lists each outside company either side talks to, with its requests before → after and the user data it receives: new kinds in red with a **+**, kinds it already received in grey, and kinds it no longer gets struck through.

*[Screenshot: Every third party: AppsFlyer, marked New, with 0 → 1 requests and + Advertising ID and + User ID in red; Mixpanel with 1 → 1 requests, + Email address in red and User ID in grey. The selected Mixpanel's detail lists Email address as New and User ID as before and after, with request #12003 on 2.3 and #12019 on 2.4.. Which companies receive what, before and after.]*

> **Check the privacy labels:**
> A new third party or a new kind of data usually means the App Store privacy label and the Play data safety form need updating. The [Privacy Map](https://docs.subcloak.com/cloak-proxy/insights/privacy-map/) shows where in each request the data travels.

## Options

| Option | Default | What it does |
|---|---|---|
| **Slower when the median is** | 1.5× the old one | How much slower an endpoint must be: 1.25×, 1.5×, 2× or 3× |
| **And at least** | 100 ms slower | Ignore small differences in fast calls: 25, 50, 100, 250 or 500 ms |
| **Compare request bodies** | On | Also report fields the app added to, removed from or retyped in what it sends |
| **Compare third parties** | On | Run the Privacy Map's scanner on each side |
| **Include images, scripts and fonts** | Off | Compare static files as endpoints too |
| **Ignore Fields…** | None | JSONPaths to leave out, one per line, such as `$.meta.generatedAt` or `$.items[*].views`. A path covers the fields inside it, and `$.items[3].x` is read as `$.items[*].x` |
| **Match Hosts…** | None | When the sides use different hosts, pairs such as `api.staging.shopapp.example = api.shopapp.example` (the after host, then the before host), so their endpoints line up |
| **App's Own Domains…** | Guessed | Registrable domains that belong to the app; everything else is a third party |

The sides and options are remembered. Kept traffic is stored on this Mac (only the newest copy), through a store that self-test and gallery runs redirect to a throwaway folder.

## Export

The share menu exports the comparison as:

- **Markdown** — a table of both sides' totals (requests, endpoints, hosts, errors, median time, data received), then a section per kind with each difference's severity, endpoint, field, before and after values, and request numbers on both sides. It reads well as a pull request comment.
- **JSON** — the whole comparison as data: both sides' totals, every endpoint, every difference and every third party.

## From an AI assistant

The [MCP server](https://docs.subcloak.com/cloak-proxy/automation/mcp-server/)'s **`release_diff`** tool compares two sides. Each side is an object: `session_path` for a saved `.cloaksession` or `.har` file, or nothing for the captured traffic, narrowed with `app`, `app_version`, `host`, `ids`, `from_id` and `to_id`. It returns both sides' totals, the counts by severity and kind, each difference (severity, kind, endpoint, field path, before, after, a sentence and the request ids on both sides) and the third parties whose data changed — or the comparison as `markdown` or `json`. `min_severity`, `category` and `limit` narrow the list; the options above are arguments too.

```text title="Ask your assistant"
"Compare ShopApp 2.3 with 2.4 in the captured traffic and tell me which changes could break the old app."
```

See the [MCP tools reference](https://docs.subcloak.com/cloak-proxy/automation/mcp-tools-reference/#release_diff) for every argument.

## In CI

`cloakctl diff` compares two recordings and exits with `2` when a difference matches `--fail-on`, so a build can fail when a release changes the API in a way you care about.

```bash title="Terminal"
cloakctl diff --before baseline.cloaksession --after e2e.har --fail-on high --out release-diff.md
```

| Option | What it does |
|---|---|
| `--before FILE` / `--after FILE` | The older and newer recordings: `.cloaksession` or `.har` (both required) |
| `--session FILE` with `--before-version V --after-version V` | One recording with both builds in it, split by the app version in the User-Agent |
| `--app NAME` / `--host HOST` | Compare only this app's or host's requests on both sides |
| `--host-map NEW=OLD` | Line up hosts that differ between the recordings (repeatable) |
| `--first-party DOMAIN` | The app's own domains (repeatable). Default: guessed |
| `--ignore-field JSONPATH` | Leave a field out (repeatable) |
| `--slower-ratio 1.5` / `--slower-min-ms 100` | When an endpoint counts as slower |
| `--include-static` | Also compare images, scripts, styles and fonts |
| `--no-third-parties` | Don't compare third parties |
| `--format markdown\|json` | The output format (default `markdown`) |
| `--out FILE` | Write the comparison to a file instead of printing it |
| `--fail-on RULE` | When to exit `2`: a severity (`high`, `medium`, `low` — that or worse), kinds of difference (`endpoints`, `fields`, `statuses`, `timing`, `requests`, `third-parties`), single differences (`field-removed`, `third-party-added`…), comma-separated, or `never` (the default) |

| Exit code | Meaning |
|---|---|
| `0` | No difference matched `--fail-on` |
| `1` | A usage error, such as a missing or unreadable file, or a side with no requests |
| `2` | At least one difference matched `--fail-on` |

A typical setup keeps a recording of the end-to-end tests from the main branch as the baseline, and compares each pull request's recording with it. Record both with [`cloakctl run --record`](https://docs.subcloak.com/cloak-proxy/automation/headless-and-ci/):

```yaml title=".github/workflows/release-diff.yml"
name: Release diff
on: [pull_request]

jobs:
  diff:
    runs-on: [self-hosted, macOS]
    env:
      CLOAKCTL: /Applications/Cloak Proxy.app/Contents/Resources/cloakctl
    steps:
      - uses: actions/checkout@v4

      - name: Record the end-to-end tests through Cloak Proxy
        run: |
          "$CLOAKCTL" run --ssl 'api.example.com' --record pr.har --quiet -- npm run test:e2e

      - name: Compare with the main branch's recording
        run: |
          "$CLOAKCTL" diff --before baseline/main.har --after pr.har \
            --fail-on high,third-parties --out release-diff.md

      - name: Keep the comparison
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: release-diff
          path: release-diff.md
```

## Limits

- Release Diff compares what was recorded. Request counts and the "no longer called" check assume both sides cover the same steps; a shorter journey reads as missing endpoints.
- Fields come from samples. An optional field that happened not to appear on one side can read as removed or added; the detail says how many samples had it, and optional fields are never reported as high.
- Timing depends on the network, the device and the server's load. Record both sides in the same conditions, and repeat the journey when a difference is small.
- Only JSON bodies are compared field by field. Bodies cut short when captured, and bodies over 4 MB, aren't sampled.
- Endpoints line up by method, host and path template. An endpoint that moved to a new path reads as one removed and one added.
- Request numbers are each recording's own. Sessions saved by Cloak Proxy 0.3 or earlier store times to the whole second, so their durations are approximate; save them again to keep milliseconds.

## Related

  - [Schema changes](https://docs.subcloak.com/cloak-proxy/insights/schema-changes/): See when an endpoint's JSON shape changes between responses in one session.
  - [Contract checking](https://docs.subcloak.com/cloak-proxy/testing/contract-checking/): Check every request and response against your OpenAPI spec.
  - [Privacy Map](https://docs.subcloak.com/cloak-proxy/insights/privacy-map/): Which outside companies receive an app's users' data, and where.
  - [Compare requests](https://docs.subcloak.com/cloak-proxy/inspect/compare-requests/): Diff two requests and responses side by side.
  - [Compare environments](https://docs.subcloak.com/cloak-proxy/testing/compare-environments/): Send captured requests to another environment and compare the answers.