Who reads the error? Designing APIs for agents

When an API returns an error, someone or something decides what to do next. For most of the history of web APIs, that decision ended with a developer reading a message in a terminal. In agent systems, the first reader is almost never a person. It is an SDK's retry policy, then a job runner's backoff logic, then possibly a circuit breaker that decides whether an entire provider is healthy.

Those readers do not parse the message. They branch on the status code. That makes the status code the most important part of an error response, and it makes getting the class wrong far more expensive than it looks.

The one question that matters

For every failure an API can produce, ask: could retrying the identical request ever succeed?

• If yes, the failure is retryable. The status should be 429 (you are going too fast) or 5xx (something on the server side or upstream failed).
• If no, the failure is terminal. The status should be a 4xx other than 429, and the body should say what to change.

This is not a new rule. It is how HTTP status classes were designed, and it is how most official SDKs behave. The OpenAI and Anthropic client libraries, for example, retry connection errors, 408, 409, 429 and 5xx responses by default with exponential backoff, and raise immediately on other 4xx responses. Check the documentation of the SDK version you use, but the pattern is consistent across the ecosystem.

The rule is easy to state and surprisingly easy to break, because the code that produces an error often does not know which class it belongs to.

How a typo becomes an outage

Consider a request to an aggregation layer with a model id that does not exist, say a misspelling. Internally, the layer looks up the id, fails to find a provider for it, and falls through to a generic "no upstream available" path. That path was written for real upstream outages, so it returns 503 Service Unavailable.

Now follow the response through the stack:

1. The SDK sees a 503, which is retryable. It waits and retries, typically two or more times with increasing delays. Every retry fails the same way.
2. The agent's job runner receives the final exception after the SDK gives up. It classifies it as transient, because it is a 5xx, and re-queues the task for later.
3. A circuit breaker watching error rates for that provider sees a burst of 503s. It opens, and now valid requests for other models on the same route are rejected too.
4. Monitoring reports a provider incident. Someone gets paged to look at a provider that is healthy.

The actual problem, a string in a config file, is never surfaced as what it is. The fix takes seconds once found, and finding it can take hours, because every layer did exactly what it was designed to do with the information it was given.

The same pattern shows up with other terminal conditions dressed as transient ones:

• a spent budget returned as 429, so the client backs off and retries forever against a limit that will not reset on its own,
• an invalid API key that falls back to an anonymous tier and then fails later with an unrelated error,
• a request that is too large for the model's context returned as a 500.

A classification table

A useful exercise is to list every error your API can return and put each one in a column. Here is the table Gatewayz uses for its public contract:

This contract has been live in production since 2026-09-08. Two details are worth pointing out.

First, the two 402 responses share a status because they share a remedy class: money or limits must change before the request can work. They are separated by the code field, because the person who fixes each one is often different. A developer raises a per-key cap; a billing owner adds credits.

Second, rate limiting is evaluated before cap accounting. A 429 does not consume any of the key's cap, so a client that backs off correctly is not penalized for having been told to slow down.

Designing the body for a program

The status code decides whether to retry. The body should let a program decide what else to do without string matching on prose.

• A stable, machine-readable code. model_not_found is a contract. "The model you requested could not be found" is copy that someone will edit.
• The offending value, when it is safe to echo. Returning the model id that failed lets a log line be useful on its own.
• No change in class across versions. Moving an error from 503 to 400 is a fix. Moving one from 400 to 503 is a breaking change, even though the endpoint signature did not change.

Circuit breakers need clean inputs

Circuit breakers are one of the more reliable patterns for protecting agents from failing dependencies, but they are only as good as the signal they count. A breaker that counts all non-2xx responses will trip on client mistakes. A breaker that counts only 5xx and timeouts depends on the server classifying correctly.

If you run agents against any model API, it is worth configuring breakers to count only:

• connection failures and timeouts,
• 5xx responses,
• optionally, sustained 429 responses, as a signal to shed load rather than as a health signal.

And it is worth testing the upstream's classification directly. Send a request with a model id that cannot exist and check that the response is a 4xx. If it is not, your breaker will eventually report a typo as an outage.

Checklist for API authors

Before shipping any new error path:

1. Answer the retry question in writing, in the code review.
2. Make sure a fallthrough or catch-all cannot convert a client error into a server error. Generic handlers should default to 500 only for genuinely unexpected exceptions, and validation should run before them.
3. Add a test that asserts the status class, not only the message.
4. Treat any change of class as a breaking change in your changelog.

Agents read errors literally. The kindest thing an API can do for them is to be literal back.