Skip to content

Rate limits and retries

Limits are counted per API key, in a fixed one-minute window, per class of call. Each key has its own counters, so one busy key never slows another.

Class Applies to Limit per minute
read GET calls other than metrics 600
metrics the two metrics endpoints 600
metrics_long metrics calls spanning more than 7 days (also counted in metrics) 120
mutate every call that changes something 60
enroll POST /v1/enroll 10

Every response to a call that passes authentication and its scope check, successful or not, carries:

Header Meaning
RateLimit-Limit The limit of the class this call counted against.
RateLimit-Remaining Calls left in the current window.
RateLimit-Reset Seconds until the window resets.

When a call is over the limit it answers 429 with a rate_limited problem — its class field names the class — and a Retry-After header with the seconds to wait. Wait that long, then retry; do not retry in a tight loop.

Every response also carries an X-Request-Id, repeated as request_id in error bodies. Log it with your own request logs, and quote it when you ask for help.

Retry 429, 500 and 503 responses, waiting for Retry-After when it is there and backing off otherwise. A GET, a DELETE and an acknowledge are safe to repeat: deleting a server twice answers 404 the second time, and acknowledging an already acknowledged incident answers 200.

Creating a server is the call to protect. Send an Idempotency-Key header (up to 64 characters) on POST /v1/servers:

  • Repeating the call with the same key and body within 24 hours returns 200 with the same server, no agent_api_key or install_command (both fields are omitted from the JSON) and a recovery hint, POST /v1/servers/{id}/agent-key:rotate. No second server is created. If the first response was lost with the agent key in it, call that endpoint to mint a new key.
  • The same key with a different body answers 422 idempotency_key_reused.
  • If Monitorable cannot check the key, the create fails with 503 idempotency_unavailable and nothing is created: retry it. The same answer comes back, with a Retry-After, while a request with that key is still running.
  • A replay counts against the rate limit like any other POST (the mutate class).

Monitorable never stores an agent key to replay it, which is why a replay returns none.

POST /v1/enroll ignores Idempotency-Key: its server_ref is what prevents duplicates (see Add servers without the dashboard). Its limit is 10 a minute per key; the installer retries for about four minutes, so roughly 40–50 hosts that boot at the same moment finish with one key. Stagger larger simultaneous boots, or give groups of hosts their own enrollment key.