# Monitors (on Subcloak's servers)

> Run a team API request on a schedule from Subcloak's servers, check it with assertions, and alert Slack, PagerDuty, Opsgenie, a webhook or email. Not end-to-end encrypted.

Source: https://docs.subcloak.com/cloak-proxy/team/monitors/

Added in Cloak Proxy 0.7.0.

{/* L9 draft. Sources: PLAN 1.1 (monitors row), 7.11, 7.14 (server-side policy), 8.4, 9.5 page 9, 6.8 (lapse); T14. Re-check labels against L5/L8. */}

A team **monitor** sends one of your API client requests from Subcloak's servers on a schedule, checks the response, and alerts your team when it starts or stops failing. It runs while every Mac is closed.

> *** Not end-to-end encrypted:**
> Monitors run on Subcloak's servers, which must read them to run them. Subcloak can read the request (its URL, headers and body), any secret you include in it, and every response. Use a low-privilege token made for monitoring.

These are separate from the API client's [local monitors](https://docs.subcloak.com/cloak-proxy/api-client/run/monitors-and-webhooks/), which run on your Mac and send nothing to Subcloak.

## Publish a monitor

{/* PLAN 7.11 publishing; T14 */}

1. In a team workspace, open a request's menu and choose **Monitor on Subcloak's Servers\*…**. The sheet starts with the warning above. A request that changed on another Mac and isn't reviewed here yet, or one in a workspace you don't trust, can't be monitored until you review the change (**Settings › Team**, **Review…** next to Sync) or trust the workspace, as it gets no vault values when you send it.

2. Choose:
   - **how often** it runs: every 5 minutes at the most, every 24 hours at the least;
   - its **assertions**: the status code, a latency limit, and text the body contains;
   - how many failures in a row (1 to 5) raise an alert, and whether to send a recovery alert; an admin also chooses its **alert routes** here;
   - which **secrets** to include. Each [vault](https://docs.subcloak.com/cloak-proxy/team/vault/) reference needs its own tick, **Send this secret to Subcloak's servers**. A reference you don't tick can't be published: Subcloak's servers have no team keys to fill it in, so tick it or take it out. A vault item limited to some hosts is only sent to a monitor that calls one of them. A monitor's URL must be plain: no vault reference, user name or backslash before its path, and no space, tab or line break anywhere.

3. Publish. Cloak Proxy sends a readable copy of the request to Subcloak's servers. The team workspace itself stays end-to-end encrypted.

The copy on our servers is encrypted at rest with Subcloak's own key, which isn't end-to-end encryption: our servers can decrypt it to run it.

## How it runs

{/* PLAN 7.11 running, SSRF guard */}

| | |
|---|---|
| Timeout | 10 seconds |
| Response read | The first 1 MB |
| Redirects | Up to 3 on the same host (scheme, host and port), each checked again. A redirect to another host isn't followed, so nothing in the request (headers, path or query) reaches it, and the run fails |
| Addresses | `http` and `https` only, ports 80, 443 and 1024 to 65535. Private, local and reserved addresses, `localhost`, `*.local`, `*.internal`, `metadata.*` and Subcloak's own hosts are refused |

Monitor only systems your team is allowed to test, at the intervals allowed. Monitors aren't for load testing, scanning or attacks. See the [terms](https://subcloak.com/terms#team-cloud).

## Results and alerts

{/* PLAN 7.11 results, alerts */}

Each run's result is kept for 30 days: the status, the latency and which assertions failed. No part of the response body is kept, so a target that echoes what the monitor sent can't show a secret to every member. Results show in **Monitors** under **Team: Acme**, and on the console's **Monitors\*** page (**Results** shows the last 50 runs).

Alerts fire when a monitor starts or stops failing:

| Route | What's sent |
|---|---|
| **Slack** | An incoming-webhook message, such as "Monitor Checkout API failing: status 503 (expected 200)" |
| **PagerDuty** | An Events API v2 trigger, and a resolve on recovery, with the monitor's id as the dedup key |
| **Opsgenie** | An alert (US or EU region) with the monitor's id as its alias, closed on recovery |
| **Webhook** | A request to a URL you choose |
| **Email** | An email to the members you choose |

An admin adds routes under **Integrations** on the console's **Monitors\*** page, then chooses them for a monitor in the app's monitor sheet or with **Alerts…** on the console's **Monitors\*** page. A monitor without a route records its results but alerts no one. Secrets such as webhook URLs and API keys are write-only: you can replace them but not read them back. Each has **Send test**. Alert content goes to the service you chose, under its terms.

## Limits

{/* PLAN 8.4 */}

| Limit (per team) | Value |
|---|---|
| Monitors | 5 for each seat, at most 100 |
| Interval | Every 5 minutes or longer |
| Alerts | 500 a month |
| Results | Kept 30 days |

## Who can change a monitor

{/* R3D-H2 fix 4, R4D-L6; hosted/durable.ts canChangeMonitor, patchMonitor */}

- Only the member who published a monitor, and admins, can edit, pause or delete it. The console can pause, resume and delete a monitor and choose its alert routes; what a monitor sends changes only from a Mac. Only admins choose alert routes.
- The request belongs to whoever last supplied it, along with the secrets in it. If someone else (an admin, for example) changes the method, URL, headers, body or secrets, they **replace the whole request**: they enter the URL again, and nothing from the old request is kept, its secrets included. The same goes for its assertions, because checks on the answer could reveal what the request sent: only the request's owner changes them, or someone replacing the request in the same edit. The monitor then sends only secrets the new owner can open, and the request is theirs.
- When a member is removed, monitors whose request sends their team vault secrets are paused, and admins get an email. Such a monitor resumes only after someone replaces its request with secrets they hold. Change the removed member's secrets at their source too.

## Pause, delete and policy

{/* PLAN 7.11, 7.14, 6.8 */}

- The console's **Monitors\*** page has **Pause** and **Delete** for each monitor.
- The policy **Server-side features (monitors, hosted mocks)** set to **blocked** makes our server refuse new monitors and changes to existing ones, and stops every monitor from running until it's allowed again. Pausing and deleting still work.
- Monitors stop when the team's subscription becomes read-only.
- A monitor used for scanning or attacks breaks the terms; we ask the team to delete it, and may end a team that keeps doing it. To report abuse, email `abuse@subcloak.com`.

## Related

  - [Hosted mocks](https://docs.subcloak.com/cloak-proxy/team/hosted-mocks/): The other feature that runs on our servers.
  - [What Subcloak can and can't see](https://docs.subcloak.com/cloak-proxy/team/what-subcloak-can-see/): Everything our servers hold for a team.
  - [Policies](https://docs.subcloak.com/cloak-proxy/team/policies/): Block server-side features for the team.