Learn
Notes on running agents in production
How an inference layer should behave when the caller is software: which model runs, what an error means, when spending stops, and how to measure any of it.
New here?
Start here
10 short steps, about ten minutes in total, from what inference is to your first request. Written for founders, investors and developers new to AI infrastructure.
- 1. What Gatewayz is
- 2. Inference
- 3. Compute
- 4. Tokens and pricing
- 5. Models and providers
- 6. APIs, endpoints and keys
- 7. Agents
- 8. Routing
- 9. Security
- 10. First request
The foundations, in full
One complete guide for each Start here step, in reading order.
Step 1
What is Gatewayz? One API key for many AI models
What Gatewayz is: one API key and one neutral endpoint for AI models from several providers. What it does, what it does not do, and the trade-offs.
9 min read
Step 2
What is AI inference? Training vs inference explained
What AI inference is, how it differs from training, what happens between request and response, and why inference is the recurring cost of using AI.
9 min read
Step 3
What is AI compute? CPUs, GPUs and who owns them
What AI compute is, why models run on GPUs instead of CPUs, who owns the hardware, and how that ownership shapes what inference costs you.
10 min read
Step 4
What are AI tokens? How LLM usage is priced
What AI tokens are, how input and output tokens and the context window work, how per-token pricing adds up, and practical ways to control LLM cost.
9 min read
Step 5
AI models and providers: snapshots, aliases, catalogs
What an AI model is, what a model provider does, and how dated snapshots and aliases work, with the providers in the Gatewayz catalog today.
10 min read
Step 6
What is an API, an endpoint and an API key?
A plain explanation of APIs, endpoints and API keys for AI models: HTTP request anatomy, Chat Completions vs Messages formats, and status codes.
10 min read
Step 7
What is an AI agent? How agents change inference
What an AI agent is: a model, tools and a loop. Why agents make many unattended calls, and why machine-readable errors and spend caps matter.
10 min read
Step 8
What is an AI gateway? Why model routing matters
What an AI gateway does: one integration for models from several providers, switching models by changing one string, failover, and why it never substitutes
8 min read
Step 9
How Gatewayz keeps your API keys and data secure
How Gatewayz protects API keys and what request data it stores, for how long, what it cannot protect, and what you should do on your side.
9 min read
Step 10
Your first Gatewayz API request in five minutes
Make your first Gatewayz API request: get a key, check model ids, call chat completions with curl or the OpenAI Python SDK, and read common errors.
8 min read
The category
What an inference layer is, and what it is responsible for.
Neutral by design
Running the model you asked for, and only that model.
Built for software, not people
Errors, limits and streams that an unattended agent can act on.
Spend ceilings for unattended agents
Rate limits slow a runaway loop down; a per-key request cap stops it. When the cap is spent the answer must be 402, not 429, and the two kinds of 402 must be told apart by code.
6 min read
Streaming failures agents can detect
Once a 200 has been flushed, an HTTP error is impossible. A stream that simply stops looks like a short success, so a failed stream has to end with an explicit error event.
6 min read
Who reads the error? Designing APIs for agents
Status codes are control flow for SDKs, retry loops and circuit breakers. A terminal problem reported with a retryable status turns a typo into an apparent outage.
7 min read
Measured, not claimed
How to test caching, latency and cost so the result means something.
How to benchmark a gateway honestly
Back-to-back runs against the direct provider, percentiles instead of averages, and rules for when a latency or cost claim may be published at all.
7 min read
Prompt caching through a gateway
cache_control has to survive the trip through the middle layer, and a small test prompt can prove nothing because below the cacheable minimum the usage fields are absent, not zero.
7 min read
Guides
Step-by-step setup for specific tools.
Reference
Short definitions of the terms these articles use, from aliases and dated snapshots to retryable statuses and TTFT.
Open the glossary →