Last updated October 13, 2026. Pricing, licenses, and feature claims were checked against each vendor’s primary docs and pricing pages on that date.
Enterprise teams evaluating LLM gateways are usually buying cost control and blast-radius limits, not another SDK wrapper. A gateway sits between apps and model providers so you can enforce virtual keys, budgets, routing, fallbacks, and audit logs without rewriting every client.
This guide ranks nine gateways on cost shape, control surface, self-host options, and ops burden for production AI teams. Placement is unpaid, per our editorial standards and disclosure policy.
It sits under the AgentOps tools map and the AI Gateways category, next to our observability tools roundup and open source LLM observability shortlist.
TL;DR
- Best managed control plane: Portkey (Production $49/mo for 100k logs; MIT OSS gateway available).
- Best self-host default: LiteLLM (MIT core free; Enterprise priced by annual request capacity, never per token).
- Best model catalog with zero ops: OpenRouter (provider rates pass through; 5.5% credit-purchase fee; BYOK free to $25k/mo list-price inference on PAYG).
- Best regulated / VPC story: TrueFoundry (Pro $25/user/mo; Enterprise adds VPC, on-prem, air-gap).
- Best if Kong already fronts APIs: Kong AI Gateway on Konnect (model proxy and request meters on Plus).
- Best free Cloudflare front door: Cloudflare AI Gateway (core free; 5% only on Unified Billing credits).
- Best high-throughput OSS proxy: Bifrost by Maxim (Apache-2.0; Enterprise custom).
- Best zero-markup managed path: Vercel AI Gateway (0% token markup including BYOK; budgets on system credentials).
- Observability-first caveat: Helicone still ships a gateway, but Mintlify acquired it on March 3, 2026 and put the product in maintenance mode.
The 9 Best LLM Gateways at a Glance
Pick by the control job you need, not by marketing checklists. The table is the decision surface; each section below explains when that gateway is rational.
| Gateway | Best for | Self-host | Entry price (gateway) | Control highlights |
|---|---|---|---|---|
| Portkey | Managed routing + guardrails + virtual keys | Yes (MIT gateway) | Dev free (10k logs); Production $49/mo | Fallbacks, budgets, RBAC, prompts |
| LiteLLM | Self-host control with wide provider coverage | Yes (MIT) | $0 OSS; Enterprise by capacity | Virtual keys, budgets, spend tags |
| OpenRouter | One bill across a large model catalog | No | 5.5% on credit buys ($0.80 min) | Provider fallbacks, BYOK allowance |
| TrueFoundry | Enterprise VPC / air-gap with seat pricing | Enterprise paths | Dev $0; Pro $25/user/mo | Budgets, guardrails, MCP gateway |
| Kong AI Gateway | Kong shops governing LLM + MCP traffic | Yes (Gateway OSS + plugins) | Konnect from ~$25/mo + AI meters | Plugins, rate limits, cost meters |
| Cloudflare AI Gateway | Free analytics / cache / rate limits on CF | No | $0 core; 5% Unified Billing | Caching, retries, fallbacks |
| Bifrost (Maxim) | High-RPS Apache-2.0 proxy you run | Yes (Apache-2.0) | $0 OSS; Enterprise custom | Virtual keys, budgets, MCP gateway |
| Vercel AI Gateway | Zero markup + budgets without running a proxy | No | 0% token markup | Failover, request logs, spend caps |
| Helicone | Gateway + deep request logs (legacy) | Yes (Apache-2.0) | Hobby free; Pro $79/mo | Caching, fallbacks, 0% model markup |
How We Chose These Nine Gateways
We scored gateways on jobs enterprise platform teams actually buy: spend ceilings, key isolation, routing and failover, observability hooks, self-host or residency options, and a price you can model without a sales call.
- Primary sources only: pricing and feature claims come from vendor pricing pages and docs fetched October 13, 2026.
- Cost shape: flat SaaS fee, percent of credits, seat meters, model-proxy meters, or self-host ops cost.
- Control surface: virtual keys, budgets, RPM/TPM limits, allowlists, guardrails, audit logs, SSO.
- Ops burden: managed SaaS versus Docker/Kubernetes you run versus hybrid VPC.
- Out of scope: pure observability suites without a gateway path (covered in observability tools and Langfuse alternatives).
1. Portkey: Managed Governance Without Building a Proxy
Portkey is the managed control plane for teams that want universal API routing, virtual keys, guardrails, and prompt management without staffing a gateway SRE rotation.
Developer is free forever for 10k recorded logs per month with short retention. Production is $49/mo for 100k logs, then $9 per additional 100k up to 3M, per Portkey pricing. Enterprise is custom for VPC hosting, SSO, BAAs, and higher retention.
The open-source gateway is MIT-licensed for self-hosting with universal API, retries, routing, fallbacks, and basic dashboards, per Portkey and AI Gateway features. Pay Portkey for recorded logs and governance packaging, not for a markup on model tokens.
Pick Portkey when you want a production SaaS gateway fast, with virtual keys and guardrails, and you can live with log-metered pricing.
2. LiteLLM: The Self-Host Default for Virtual Keys and Budgets
LiteLLM is the open-source gateway most platform teams try first when they need OpenAI-compatible routing across many providers plus virtual keys, budgets, and spend attribution inside their own VPC.
The MIT open-source gateway is free to self-host in production. LiteLLM Enterprise adds SSO, SCIM, audit logs, secret-manager hooks, and support, priced by annual gateway request capacity and architecture, never per token, per LiteLLM pricing and Enterprise docs.
Virtual keys, team budgets, RPM/TPM limits, and tag-based spend tracking are the control surface that makes LiteLLM an enterprise gateway rather than a thin proxy, per LiteLLM docs. Your ops team owns uptime; the license fee is optional until SSO and audit requirements appear.
Pick LiteLLM when you will run the gateway yourself and want the widest provider coverage with budgets and virtual keys on day one.
3. OpenRouter: One API Across a Huge Catalog
OpenRouter is a managed unified API over a large multi-provider model catalog. It wins when time-to-many-models matters more than self-hosting the control plane.
OpenRouter passes through provider inference prices without markup and charges a 5.5% fee ($0.80 minimum) when you buy credits, or 5% for crypto, per OpenRouter FAQ. Automatic provider fallbacks and routing variants such as :nitro and :floor are built in.
Bring-your-own-key (BYOK) includes $25,000 per month of list-price inference with no BYOK fee on pay-as-you-go, then 5% of equivalent OpenRouter cost above that allowance, per FAQ pricing and BYOK docs. Model spend is transparent; the platform fee sits on credit purchases and over-allowance BYOK.
Pick OpenRouter when you want hundreds of models on one key and will accept a percent fee instead of running LiteLLM or Portkey yourself.
4. TrueFoundry: Seat-Priced Gateway With Hard Residency Paths
TrueFoundry packages an LLM and MCP gateway with budgets, guardrails, and deployment modes that include managed SaaS, VPC, on-prem, and air-gapped Enterprise options.
Developer is free for up to 3 users with 10k requests and tool calls per user per month. Pro is $25 per user per month with 20k pooled requests per user and $20 per additional 100k, per TrueFoundry pricing.
Enterprise adds custom retention, external IdP mapping, third-party guardrails, VPC / on-prem / air-gap, and SLAs. Seats and traffic are separate meters, so a small team can push high request volume without buying fake headcount.
Pick TrueFoundry when procurement needs a commercial gateway with clear seat pricing and a documented private-deployment path.
5. Kong AI Gateway: Govern LLM Traffic Where APIs Already Live
Kong AI Gateway is the right answer when Kong already fronts your APIs and security teams refuse a second control plane for LLM, MCP, and agent traffic.
AI features ship as Kong Gateway plugins with a Universal API through AI Proxy / AI Proxy Advanced, per Kong AI Gateway docs. Konnect Plus starts from about $25/mo plus usage and meters AI requests and distinct LLM models proxied.
On the October 2026 Kong pricing page, Plus includes 1M AI requests/mo ($200 per additional 1M, capped at 10M) and up to 5 distinct models at $100/mo per model. Advanced AI plugins and AI Gateway Manager are add-ons on Plus and included on Enterprise.
Pick Kong when platform standards already say Kong, and you need LLM policy next to the same auth, rate-limit, and observability stack.
6. Cloudflare AI Gateway: Free Front Door on the Edge
Cloudflare AI Gateway gives any Cloudflare account analytics, caching, rate limiting, retries, and model fallback with a one-line integration. It is thin on deep enterprise RBAC, strong as a free edge control layer.
Core gateway features are available on Cloudflare plans without a separate gateway SKU, per Cloudflare AI Gateway docs. Unified Billing, where Cloudflare purchases provider credits for you, carries a 5% fee on those credits.
Persistent logs are capped by Workers plan (for example 100k total on Workers Free and much higher on Paid). Use Cloudflare when the app already sits on Cloudflare and you want caching plus basic spend visibility before adopting a heavier gateway.
Pick Cloudflare as a low-friction first gateway, then graduate to LiteLLM, Portkey, or Kong when virtual-key org design outgrows the edge dashboard.
7. Bifrost: Apache-2.0 Gateway Tuned for Throughput
Bifrost, published by Maxim AI, is an Apache-2.0 LLM, MCP, and agent gateway that markets sub-millisecond overhead at multi-thousand RPS. It is the OSS pick when LiteLLM’s ops profile feels heavy and raw proxy performance matters.
OSS is free to self-host with OpenAI-compatible APIs, failover, load balancing, semantic caching, virtual keys, budgets, and MCP gateway features, per Bifrost and GitHub. Enterprise is custom and adds guardrails, cluster mode, SSO/RBAC, audit exports, and VPC / air-gap options, per Bifrost pricing.
Teams already evaluating Maxim for simulation and online evals can keep gateway and evals in one vendor family; compare that packaging in Braintrust vs Maxim vs Galileo and AI agent evaluation tools.
Pick Bifrost when you want Apache-2.0 self-host with virtual keys and budgets, and you care about gateway overhead under load.
8. Vercel AI Gateway: Zero Markup Without Running Infra
Vercel AI Gateway is a managed multi-provider gateway you can call from any infrastructure. It centralizes credentials, logs routing attempts, and enforces budgets while adding zero markup to provider token prices, including BYOK.
Budgets apply to spend billed through AI Gateway system credentials; BYOK spend is metered separately and does not count toward those budget caps, per Vercel AI Gateway docs. Failover across providers and ordered model fallbacks are first-class.
Choose Vercel when you want OpenRouter-like convenience with 0% token markup and explicit team or project budgets, without standing up LiteLLM. Confirm current credit and discount rules on Vercel AI Gateway pricing before modeling large BYOK workloads.
Pick Vercel AI Gateway for managed failover and spend caps when zero token markup matters more than self-host residency.
9. Helicone: Observability-First Gateway With a Roadmap Caveat
Helicone still ships an OpenAI-compatible AI Gateway with caching, rate limits, automatic fallbacks, and 0% markup on provider model spend. It belongs on a 2026 shortlist only with ownership context.
Hobby is free for 10k requests/mo. Pro is $79/mo and Team is $799/mo, both with usage-based charges above included requests and storage, per Helicone pricing. Gateway behavior is documented in Helicone AI Gateway docs.
Mintlify acquired Helicone on March 3, 2026. Helicone is in maintenance mode: security fixes and model support continue, new feature development does not. Mintlify has said it will help customers migrate, per Helicone joining-Mintlify post.
For greenfield observability plus gateway work, prefer Portkey, LiteLLM, or Bifrost, and read the self-host comparison in Phoenix vs Langfuse vs Helicone. Keep Helicone only if you already depend on its log UX and need continuity.
Pick Helicone only for continuity on an existing deployment. Do not start a new enterprise gateway program on a maintenance-mode product.
Decision Table: Cost Shape Versus Control Depth
Hard gates beat preference. If two gateways clear every gate, run a two-week dual-proxy bake-off on the same production traffic with identical budgets.
| Question | Prefer |
|---|---|
| Need MIT/Apache self-host with virtual keys? | LiteLLM or Bifrost |
| Need managed SaaS with guardrails fast? | Portkey Production |
| Need huge catalog, accept % fee? | OpenRouter |
| Need 0% token markup managed? | Vercel AI Gateway (or Helicone BYOK path) |
| Need VPC / air-gap on a commercial contract? | TrueFoundry Enterprise (or LiteLLM / Bifrost Enterprise) |
| Already standardized on Kong? | Kong AI Gateway |
| Already on Cloudflare and want a free edge layer? | Cloudflare AI Gateway |
| Primary need is deep request logs, not new features? | Helicone (legacy) or Portkey |
| Primary need is agent evals, gateway secondary? | Keep Maxim Bifrost + eval suite; see eval guides |
Pricing Shapes Enterprises Actually Model
Gateway bills are not interchangeable with observability unit meters. Model the gateway fee separately from provider tokens, then stress-test fallback traffic that doubles request counts during incidents.
- Flat SaaS + logs: Portkey Production $49/mo for 100k logs; Helicone Pro $79/mo platform fee before overages.
- Percent of credits: OpenRouter 5.5% on card credit purchases; Cloudflare 5% on Unified Billing credits only.
- Seats + overage: TrueFoundry Pro $25/user/mo with $20/100k request overage.
- Infra you run: LiteLLM OSS and Bifrost OSS are $0 license; add compute, Postgres, and on-call cost.
- API-platform meters: Kong Konnect Plus charges AI request volume and per distinct model proxied.
- Zero markup managed: Vercel AI Gateway states 0% markup on provider tokens including BYOK.
# illustrative monthly gateway fee (provider tokens excluded)
portkey_over=(max(0,logs-100000)//100000)*9
portkey_prod=49+portkey_over
openrouter_fee=credit_purchases*0.055
tf_included=users*20000
tf_over=(max(0,reqs-tf_included)//100000)*20
truefoundry_pro=users*25+tf_over
What Searchers and LLMs Usually Get Wrong
Public roundups often treat every “AI gateway” as interchangeable, skip percent-fee math, or still rank Helicone as a greenfield leader without the Mintlify maintenance-mode note.
- Gateway versus observability: Langfuse-class tools observe; gateways authorize, route, and cap spend. Many teams need both.
- BYOK is not always free: OpenRouter’s BYOK allowance is large but not unlimited; Vercel budgets may ignore BYOK spend.
- Self-host is not free: LiteLLM and Bifrost remove license fees; they do not remove SRE time.
- Kong is not “just another SaaS gateway”: it is rational mainly when Kong is already the API standard.
Primary, Secondary, and Long-Tail Questions This Guide Answers
Beyond the head term LLM gateways, teams ask about cost control, virtual keys, OpenRouter fees, and Kong versus specialist proxies. Those questions map to the nine tools above.
- Primary: llm gateways, best llm gateway, ai gateway
- Secondary: portkey vs litellm, openrouter vs portkey, llm gateway pricing, enterprise ai gateway
- Long-tail / LSI: virtual keys llm, llm budget controls, cloudflare ai gateway pricing, kong ai gateway cost, bifrost gateway, vercel ai gateway markup
- Fan-out: self-host versus managed, percent fees versus flat SaaS, MCP gateway overlap. Related reading: AI agent frameworks and AgentOps tools.
Frequently Asked Questions
What is an LLM gateway in an enterprise stack?
An LLM gateway is a control proxy between applications and model providers. It centralizes credentials, enforces virtual keys and budgets, routes or fails over across models, and emits logs or metrics. Platform teams buy it for cost ceilings and blast-radius limits, not for model quality itself.
How is gateway pricing different from model token pricing?
Model providers bill tokens or requests. Gateways add a separate fee shape: flat SaaS plus logs, a percent of credits, seats plus overage, API-platform meters, or zero license with self-host ops cost. Always model provider tokens and the gateway fee as two lines.
Should we self-host LiteLLM or buy Portkey?
Self-host LiteLLM when residency, custom plugins, and lowest license cost matter and you already have gateway operators on call. Buy Portkey when you want managed fallbacks, guardrails, and virtual keys quickly, and you will accept log-metered SaaS pricing instead of running and patching the proxy yourself.
Is OpenRouter cheaper than a self-hosted gateway?
OpenRouter is often cheaper on engineering time and more expensive on percent fees at high credit volume. Run the 5.5% credit fee against your monthly model spend, then compare it to LiteLLM or Bifrost compute plus on-call before you decide.
Why is Helicone ranked last for new programs?
Helicone still offers a capable gateway and deep logs, but Mintlify acquired it on March 3, 2026 and placed the product in maintenance mode. Security fixes continue; a greenfield enterprise roadmap does not. Prefer Portkey, LiteLLM, Bifrost, or Kong for new builds.
Related Guides
If LiteLLM is already your default and you are weighing a switch, our LiteLLM alternatives guide covers eight production routing options, from managed gateways to self-hosted proxies.
LLM gateways sit on the model path. MCP is the sibling control plane for tools. MCP Explained walks through servers, gateways, and security for that layer.
MCP tool access is a sibling control plane to LLM routing. When agents need governed Model Context Protocol servers, start with the best MCP gateways guide.
Choosing among Portkey, LiteLLM, and OpenRouter specifically? See our head-to-head on Portkey vs LiteLLM vs OpenRouter for stack fit, BYOK, and cost shape.
- AgentOps in 2026: The Complete Map of Observability and Eval Tools
- 12 Best AI Agent Observability Tools in 2026
- 8 Best Open Source LLM Observability Tools You Can Self-Host
- Arize Phoenix vs Langfuse vs Helicone
- 7 Best Langfuse Alternatives in 2026
- 10 Best AI Agent Evaluation Tools in 2026
- 12 Best AI Agent Frameworks in 2026
- AI Gateways category hub
Questions about this comparison or a correction on a dated price? Contact the AgentStack desks. Editorial policy lives on our standards and disclosure pages.