Spend ceilings for unattended agents

Any agent that calls a model in a loop can, given the right bug, call it far more often than intended. A retry that never gives up, a planner that keeps re-planning, a queue consumer that re-enqueues its own failures: these are ordinary mistakes, and they are expensive when each iteration is a paid request.

A human would notice after a few minutes. An agent running overnight will not. The protection has to live in the layer that serves the requests, and it has to produce a signal the agent can act on without interpretation.

Rate limits are not spend limits

It is tempting to treat rate limiting as cost protection. It is not, for two reasons.

Rate limits reset. A limit of some number of requests per minute caps the speed of spending, not the total. A loop that runs all night at the permitted rate spends all night.

Rate limits are meant to be retried. A 429 Too Many Requests tells the client "slow down and try again". Well-behaved SDKs do exactly that, automatically. That is correct behavior for rate limits and the wrong behavior for an exhausted budget.

What operators need is a ceiling: a total amount a given key may use, after which requests stop until someone deliberately changes something.

Per-key request caps

Scoping ceilings to keys, rather than only to accounts, matches how agents are deployed. A team will often give each agent, environment or customer integration its own key. A per-key cap means:

• a bug in one agent exhausts that agent's allowance and nothing else,
• a staging key cannot spend like a production key,
• a key handed to a contractor or a third-party tool has a known maximum exposure.

The cap is part of the key's configuration, not something the client enforces. Client-side counters are useful for reporting, but they reset when the process restarts, and a process that restarts in a loop is one of the failure modes the cap exists for.

Why a spent cap must be 402, not 429

When a key's cap is spent, the response has to communicate one thing clearly: retrying will not help. Nothing about the passage of time will make the next identical request succeed. A person has to raise the cap, or the agent has to switch to a different key.

That is a terminal condition, and terminal conditions belong in the 4xx range outside 429. The natural fit is 402 Payment Required: the request is well-formed and authenticated, but the limits attached to it do not permit it.

Consider what happens if a spent cap is returned as 429 instead:

1. The SDK retries with backoff, several times.
2. The agent's own retry logic, which treats 429 as transient, schedules another attempt.
3. The loop continues indefinitely, making requests that all fail.
4. Nothing escalates, because from the client's point of view it is being politely rate limited.

The cap has stopped the spending, but the agent is now stuck in a silent, permanent retry loop, and nobody is told. Returning 402 turns the same situation into an immediate, non-retryable failure that surfaces in the agent's error handling and in whatever alerting watches it.

Gatewayz returns 402 with code request_cap_exhausted for this case. The general principle is covered in Who reads the error?.

Two kinds of 402

There is a second condition with the same remedy class: the account has run out of credits. It is also terminal, and it also returns 402. The difference is who fixes it.

These are separated by the code field in the error body, not by different status codes. That keeps HTTP semantics honest, since both are "the request cannot proceed until a limit or balance changes", while letting a program route each one correctly. An agent that sees request_cap_exhausted might fail over to a secondary key it has been given. An agent that sees insufficient_credits should stop entirely, because every key on the account is affected.

Ordering: rate limits before caps

A detail that is easy to get wrong: in what order are the checks applied?

In Gatewayz, rate limiting fires before cap accounting. A request rejected with 429 does not consume any of the key's cap. This matters because an agent that is correctly backing off from rate limits should not be draining its allowance by being told to wait. If the order were reversed, a burst of throttled retries could exhaust a cap without a single request reaching a model.

Handling this in agent code

A minimal policy for an unattended agent:

from openai import OpenAI, APIStatusError

client = OpenAI(
    base_url="https://api.gatewayz.ai/v1",
    api_key=GATEWAYZ_KEY,
)

try:
    resp = client.chat.completions.create(
        model="anthropic/claude-sonnet-4-6",
        messages=[{"role": "user", "content": task}],
    )
except APIStatusError as err:
    if err.status_code == 402:
        body = err.body if isinstance(err.body, dict) else {}
        code = (body.get("error") or {}).get("code") or body.get("code")
        if code == "request_cap_exhausted":
            stop_agent(reason="key cap spent", escalate_to="key owner")
        else:
            stop_agent(reason="account out of credits", escalate_to="billing")
    raise

The exact location of the code field depends on how your SDK exposes error bodies, so inspect a real 402 response once and adjust the lookup. The shape of the policy is what matters:

• 402 is never retried.
• The code decides who is told.
• The agent stops and escalates, rather than looping.

Setting the ceiling

A cap is most useful when it reflects an expectation you could state out loud: "this nightly job makes roughly this many calls; if it makes several times that, something is wrong." Set the cap somewhat above normal usage, so ordinary variation does not trip it, and well below the amount you would be unhappy to discover on an invoice.

Then test it. Create a key with a very small cap, run the agent against it, and confirm that the agent stops, reports the right reason, and does not retry. A ceiling that has never been hit in a test is a ceiling you are trusting on faith.