AI
Everyone wants to be the AI gateway
Anh-Tho Chuong•Aug 17•5 min read
Stripe is reportedly nearing a deal to buy OpenRouter for more than $7 billion. Three months ago, OpenRouter had just raised a $113 million Series B.
Meanwhile, Palo Alto Networks paid $140 million for Portkey and made it the AI gateway inside Prisma AIRS. Stripe wants the layer that brokers model demand. Palo Alto Networks wants the layer that governs agent traffic. Both are buying the middle.
The deals expose the category’s identity problem: products called AI gateways often do different jobs.
A gateway is a place in the request path, not a product category. That explains why Ramp, Merge, Vercel, Cloudflare and AWS can all launch an “AI gateway” without building the same thing.
An AI gateway sits between a caller and whatever executes the request:
Application or agent → Gateway → Model, tool, API or resource
Ollama can expose an OpenAI-compatible API, but it usually executes the model itself. It is a runtime. A gateway sits between the caller and the system executing the request.
Own that middle box and you get to make a decision before the request runs. You also see what happened afterward: cost, latency, usage, errors and demand.
What changes from product to product is the decision made in that box.
“Gateway” names the place, not the job
Model choice used to be an implementation detail. It is now a moving target. Providers change prices, models leapfrog each other, capacity disappears and new serving options show up every week. A team using several models needs somewhere to encode its preferences and fallbacks.
The same request now carries more policy than a normal API call. It may contain sensitive data. It may need to stay in a region, fit within a budget or use an approved provider. A cached answer should not be treated like a fresh model call.
Agents make the boundary even more useful. A request can now lead to an API call, a database write or a purchase. Someone needs to decide whether the agent may act, whose credentials it may use and whether the resource has been paid for.
Those are different jobs that happen to fit in the same place.
The commercial incentives are different too. A router can charge on model usage or make a developer platform stickier. A cloud gateway can pull more traffic into the cloud. An agent gateway can drive adoption of the surrounding runtime and identity stack. A payment gateway can take a fee or expand a payment network.
Much of the proliferation comes from existing companies extending their strongest asset into a newly valuable control point.
Four jobs are hiding under the same label
01. Model routing
Which model should handle this request?
OpenRouter started with unified model access. Vercel AI Gateway brings routing and fallback into Vercel’s developer workflow. Ramp Router applies Ramp’s cost-optimization instinct to model choice. Merge Gateway looks like a natural extension of Merge’s original product: one normalized API in front of a fragmented supplier base. LiteLLM and Bifrost attack the same problem from open-source proxies: one interface, many providers, with routing and spend controls attached.
They overlap, but their distribution is different. That matters as much as the routing algorithm.
02. Traffic control
Can this request run, and under which rules?
Cloudflare AI Gateway logs, caches and controls model traffic. Kong and Azure API Management are extending control planes that enterprises already use for APIs. Envoy AI Gateway, kgateway and Traefik Hub AI Gateway bring the cloud-native gateway stack into AI traffic. Portkey, now part of Palo Alto Networks, and TrueFoundry come from the AI observability and governance side. Their feature lists overlap; their routes into the customer do not.
03. Agent access
Which tools can this agent use, and on whose behalf?
Agent access is about tools rather than tokens. The gateway decides whether an agent can call Salesforce, whose identity it carries and which credentials it receives.
Amazon Bedrock AgentCore Gateway fronts tools, other agents and models. Google Agent Gateway is building an ecosystem around identity, security and governance. agentgateway provides an open-source data plane for LLM, MCP and agent-to-agent traffic. Gravitee Agent Gateway extends an established API-management stack to A2A and MCP. Some also route models. Their harder job is controlling which actions an agent may take, and on whose behalf.
04. Commercial control
What may this customer consume, and on which terms?
Cloudflare’s Monetization Gateway is one early expression of it. The gateway checks payment before returning a page, dataset, API or MCP tool. Its initial flow uses HTTP 402 and x402: the resource returns payment instructions, the caller pays and retries with proof.
This makes sense for machine-to-machine purchases. It does not replace subscriptions, negotiated prices, credits, commitments, taxes or invoicing. “Has this request been paid for?” is narrower than “what did this customer buy?”
We wouldn't rank these four groups on a single up-and-to-the-right chart. A gateway that owns more decisions also owns more latency, failure modes and switching cost. More surface area can be an advantage, but it is not a neutral measure of product quality.
The categories will blur
Features are already converging. Routers add policy and budgets. API gateways add model selection. Agent gateways front models as well as tools. Cloudflare now has both an AI Gateway and a Monetization Gateway.
Features will probably converge faster than the businesses behind them.
A product sold to a platform engineering team through an existing cloud contract behaves differently from one adopted by a developer with an API key. A router earning margin on model spend has a different incentive from an API platform using the gateway to retain traffic. The interface may look similar while the buyer, distribution and economics remain different.
If, a few years from now, the successful products have the same buyer, make money the same way and replace each other cleanly, then this map was describing a temporary phase. We expect similar feature lists wrapped around different control points and distribution advantages.
When we compare gateways, we ask five questions:
1. What traffic passes through it?
2. Which decision does it make before the request runs?
3. What advantage did the company bring from its existing business?
4. How does the gateway make money, directly or indirectly?
5. Where does the customer contract live?
Commercial control is the missing gateway decision
At Lago, we are building toward this commercial gateway layer. Runtime infrastructure is starting to absorb commercial logic without the customer agreement needed to interpret it.
A gateway needs to decide quickly how a request should run. The commercial system has to remember what the customer was promised.
Consider a customer with an annual commitment, a monthly allowance, prepaid credits and a negotiated price for one model family. A token count does not tell you what to charge. Neither does a successful x402 payment. You need the contract, the running balances and the history of adjustments.
A useful architecture separates three responsibilities:
Execution gateway. Routes the request and applies technical policy.
Commercial gateway. Checks the customer’s entitlements, balance, credits, commitments and spending rules before or during consumption.
Commercial system of record. Settles what happened into balances, invoices and revenue records.
These layers should exchange decisions and usage. They do not need to collapse into one universal gateway. Lago’s opportunity is to make commercial policy available in the request path while preserving the contract and ledger behind it.
The Lago Agent SDK is one test of this boundary. It normalizes usage from OpenAI, Anthropic, AWS Bedrock, Google Gemini and Mistral while leaving the application’s model client in place.
With Cloudflare AI Gateway, Lago can read the logs and use the cost Cloudflare actually metered. Cache hits carry zero tokens and zero cost, so they settle at zero. Cloudflare’s metered cost does not include reasoning tokens, so thinking-heavy models should be billed from token usage instead. The implementation is documented here.
Cloudflare stays responsible for execution truth. Lago applies the customer contract. We are extending the same adapter model to other execution layers, including Vercel AI Gateway, while building toward synchronous commercial decisions for use cases that need authorization before consumption.
When evaluating a gateway, ask which decision you are delegating and what becomes hard to move later.
