Last updated October 14, 2026. Pricing, licenses, ownership, and migration claims were checked against each vendor’s primary docs and pricing pages on that date.
The best LiteLLM alternatives in 2026 are Bifrost for a self-hosted drop-in, Portkey for a managed control plane, and Agent Router for Kubernetes platform teams. Kong AI Gateway, TrueFoundry, Cloudflare AI Gateway, Vercel AI Gateway, and OpenRouter each win a narrower case, from existing Kong estates to one bill across hundreds of models.
LiteLLM is an open-source LLM gateway and Python SDK that puts 140+ model providers behind one OpenAI-compatible API, with virtual keys, budgets, and spend tracking in the MIT core. Teams usually start looking for a LiteLLM proxy alternative for three reasons: the per-worker resource footprint, the March 2026 PyPI supply chain compromise, and the Enterprise license boundary around SSO and audit logs. Placement here is unpaid, per our editorial standards and disclosure policy.
Staying put is still the right call for some teams. If you run fewer than a few hundred requests per second, pin versions, and rely on LiteLLM’s long provider list, a migration buys little. This guide sits under the AgentOps hub and the AI Gateways category, next to the AgentOps tools map.
TL;DR
- Bifrost is the closest self-hosted drop-in: Apache-2.0 core, a
/litellmendpoint for LiteLLM SDK code, and a read-only migration tool that copies models, keys, teams, users, and virtual keys out of a running LiteLLM proxy. - Portkey is the managed pick: Developer is free for 10k recorded logs a month and Production is $49/mo for 100k logs. Palo Alto Networks closed its acquisition on May 29, 2026, and now sells it as Prisma AIRS AI Gateway.
- Agent Router, formerly Envoy AI Gateway, joined the Agentic AI Foundation on September 9, 2026. It is Apache 2.0, Envoy-based, and on a stable 1.x API since June 2026.
- Kong and TrueFoundry fit enterprises with existing API gateway standards or private-deployment contracts. TrueFoundry Pro is $25 per user per month with 20k pooled requests per user.
- Cloudflare, Vercel, and OpenRouter remove gateway operations entirely. Cloudflare core features are free, Vercel adds 0% token markup including BYOK, and OpenRouter charges 5.5% on card credit purchases.
- Footprint gap: a June 2026 third-party benchmark measured LiteLLM at 324 req/s and 2.3 GB peak RAM on a 2 vCPU box, against 3,100 req/s and 143 MB for Bifrost.
LiteLLM Alternatives at a Glance
Pick the alternative by who runs the gateway and which constraint already exists in your stack. Self-hosted options keep prompts and keys in your network. Managed options trade that for zero proxy operations. The table lists the incumbent first so every fee and license is compared against the setup you are leaving.
| Gateway | Best for | Self-host | License or plan entry (Oct 2026) | Virtual keys and budgets |
|---|---|---|---|---|
| LiteLLM (incumbent) | Widest provider list in your VPC | Yes | MIT core $0; Enterprise by request capacity | Yes in OSS |
| Bifrost | Self-hosted drop-in with migration tool | Yes | Apache-2.0 $0; Enterprise custom | Yes in OSS |
| Portkey | Managed governance and guardrails | Yes (OSS gateway) | Dev free; Production $49/mo | Yes; budgets on paid plans |
| Agent Router | Kubernetes and Envoy platform teams | Yes | Apache 2.0 $0 | Token QuotaPolicy, upstream credentials |
| Kong AI Gateway | Shops where Kong fronts APIs | Yes | Konnect Plus $100/mo per distinct model (max five) | Consumers, ACLs, rate limits |
| TrueFoundry | Commercial VPC, on-prem, air-gap | Enterprise | Dev $0; Pro $25/user/mo | Budget rules (1 on Dev, 10 on Pro) |
| Cloudflare AI Gateway | Free first layer on Cloudflare | No | Core free; 5% on Unified Billing credits | Spend limits by model, provider, or metadata (20 rules) |
| Vercel AI Gateway | Zero token markup, managed | No | 0% markup, BYOK on paid tier | Budgets per team, project, key, member |
| OpenRouter | Widest catalog on one bill | No | 5.5% on card credits ($0.80 min) | Guardrail budgets per member and key |
Why Teams Replace LiteLLM in 2026
Most teams leave LiteLLM because of operations and supply chain risk, with license cost a distant third. The MIT proxy charges no per-token fee, so the bill that grows is compute, Postgres, Redis, and on-call time. The March 2026 incident turned an abstract dependency risk into an audit finding for many security teams.
The resource footprint per worker
LiteLLM’s own production best practices ask for 1 vCPU and 4Gi of memory per worker, as both request and limit, and they call 4Gi a floor. The docs recommend Redis 7.0 or newer once you run more than one instance, and they route spend writes through a Redis transaction buffer above roughly 1,000 requests per second to avoid Postgres deadlocks.
A reproducible June 2026 benchmark by the author of GoModel, a competing gateway, measured LiteLLM at 30.6 ms p50 overhead, 324 req/s sustained, 2.3 GB peak RAM, and a 25.5 second cold start on an AWS c7i.large. Treat it as a hot-path footprint test with logging disabled, because it does not score features, provider coverage, or dashboards.
The March 2026 PyPI compromise
On March 24, 2026, LiteLLM versions 1.82.7 and 1.82.8 on PyPI shipped a credential stealer. LiteLLM’s security update says the packages were live for about 40 minutes before PyPI quarantined them, and that the compromise likely came through the Trivy dependency in its CI/CD scanning workflow. Version 1.82.6 is the last confirmed clean release.
The Cloud Security Alliance research note adds that version 1.82.8 dropped a litellm_init.pth file that ran on every Python startup, so any host that installed it must be treated as fully compromised. BerriAI engaged Mandiant for incident response, per Trend Micro, and LiteLLM audited every release from 1.78.0 to 1.82.6. Pinning versions and verifying hashes closes most of this risk without a migration.
The Enterprise license boundary
LiteLLM OSS covers virtual keys, teams, budgets, rate limits, and spend tags. SSO is free for up to five users, and SCIM, audit logs, secret manager hooks, and fuller org RBAC sit behind LiteLLM Enterprise, priced by annual gateway request capacity, per LiteLLM pricing. Teams that hit that boundary often compare the quote against a managed gateway before renewing.
How We Picked These Eight Alternatives
We kept gateways that can take over LiteLLM’s production job: one OpenAI-compatible endpoint, routing and fallbacks across several providers, and some form of key or spend control. Each pick had to show its pricing or license on a public page we could capture on October 14, 2026.
- Self-host versus managed: can prompts, provider keys, and logs stay inside your network, and under which plan.
- Cost controls: virtual keys, budgets, rate limits, and how the gateway itself bills you.
- Observability: request logs, metrics, and exports to tools such as Langfuse or OpenTelemetry backends.
- Guardrails: PII, content, or policy checks in the request path, and whether they cost extra.
- Multi-provider routing: fallbacks, load balancing, and provider coverage you can verify.
- Migration effort: base URL swap only, a LiteLLM import tool, or a config rewrite.
We did not run a paid load test, so there are no star ratings. Benchmarks quoted below name who ran them. For gateway pricing in a broader roundup, see the AgentOps tools map, and for logging neighbors see AI agent observability tools.
1. Bifrost: The Closest Self-Hosted Drop-In
Bifrost, published by Maxim AI, is an Apache-2.0 LLM gateway written in Go and the most direct LiteLLM proxy alternative for teams that want to keep self-hosting. It ships virtual keys, budgets, failover, load balancing, semantic caching, and an MCP gateway in the open-source build, per Bifrost.
Migration is the strongest argument. The LiteLLM migration tool reads models, keys, organizations, teams, users, and virtual keys from a running LiteLLM proxy and recreates them in Bifrost, and it never modifies the LiteLLM side. Code built on the LiteLLM SDK can point at a /litellm endpoint and keep its provider switching.
Maxim’s own benchmark page reports 0.99 ms gateway overhead against 40 ms for LiteLLM at 500 RPS on a t3.medium, and 11 microseconds at 5,000 RPS on a t3.xlarge. The independent June 2026 run above recorded 2.5 ms p50 and 18.3 ms p99 for Bifrost, so expect the tail to be wider than the vendor headline.
- Best for: platform teams leaving LiteLLM that want the same self-hosted shape with lower overhead.
- Pricing: $0 for the Apache-2.0 build; Enterprise is custom and adds guardrails, cluster mode, SSO, RBAC, and audit exports.
- Limitation: providers that exist only in LiteLLM, such as
together_aiordeepseek, must be modeled as custom OpenAI-compatible providers, and requests to unsupported providers fail.
2. Portkey: Managed Control Plane With Guardrails
Portkey is the LiteLLM alternative for teams that want virtual keys, guardrails, prompt management, and log analytics without running the proxy fleet. Its open-source gateway also self-hosts with routing, retries, fallbacks, load balancing, guardrails, and a basic dashboard, per Portkey pricing.
Developer is free for 10k recorded logs a month with 3-day log retention. Production is $49/mo for 100k recorded logs plus $9 per extra 100k up to 3M, and adds RBAC, partner guardrails, and semantic caching. Enterprise adds SSO, granular budgets and rate limits, VPC hosting, and BAAs at custom pricing.
- Best for: product teams that want a managed gateway this week and can accept log-metered pricing.
- Ownership: Palo Alto Networks completed the acquisition on May 29, 2026, and announced Prisma AIRS AI Gateway general availability on July 16, 2026.
- Limitation: log storage, multi-team RBAC, and SSO live in the managed tiers, so the OSS gateway alone does not replace LiteLLM’s built-in key and spend database.
3. Agent Router: Kubernetes-Native and Apache 2.0
Agent Router, formerly Envoy AI Gateway, is the open-source control plane for LLM and MCP traffic on Envoy, and the best fit for teams that already run Kubernetes Gateway API. Platform teams describe providers, credentials, quotas, and failover once, and applications call one OpenAI-compatible endpoint, per the Agent Router docs.
The project shipped 1.0 in June 2026 with a compatibility promise for the 1.x series, then 1.1 in August. It joined the Agentic AI Foundation on September 9, 2026, with no CRD, API, or CLI renames. Tetrate lists 11 public adopters, including Bloomberg, Tencent Cloud, and Nutanix, and 132 contributors from 21 organizations.
- Best for: SRE teams that want LLM routing to live next to existing Envoy policy, mTLS, and observability.
- Pricing: $0 under Apache 2.0; commercial support comes from vendors such as Tetrate.
- Limitation: configuration lives in Kubernetes resources such as
AIGatewayRouteandBackendSecurityPolicy, so teams without Kubernetes operators face a steeper start than LiteLLM’s single container.
Local trials are simple. The command aigw run with an OpenAI key starts a router at localhost:1975, which is enough to replay a slice of LiteLLM traffic before committing to a cluster rollout.
4. Kong AI Gateway: Reuse the API Gateway You Already Run
Kong AI Gateway is the right LiteLLM replacement when Kong already fronts your APIs and security will not approve a second gateway. AI Proxy and AI Proxy Advanced plugins add multi-provider routing, while existing Kong auth, ACLs, rate limits, and logging apply to LLM and MCP traffic, per Kong AI Gateway.
- Best for: enterprises with Kong Gateway Enterprise or Konnect already in production.
- Pricing: on Kong pricing, Konnect Plus meters each distinct proxied model at $100/mo, capped at five; Enterprise is custom.
- Limitation: advanced AI plugins are add-ons on Plus, and the per-model meter makes wide model catalogs expensive compared with LiteLLM’s $0 license.
5. TrueFoundry: Commercial Gateway With Private Deployment
TrueFoundry AI Gateway suits procurement-led teams that want LiteLLM-style routing with a signed contract, SLAs, and VPC, on-prem, or air-gapped deployment. It bundles an LLM gateway, MCP gateway, budget rules, built-in guardrails, and SSO on one platform with seat-based pricing, per TrueFoundry.
Developer is free for up to three users with 10k requests and tool calls per user per month. Pro is $25 per user per month with 20k pooled requests per user and $20 per extra 100k, per TrueFoundry pricing. A seven-person team running 1M monthly requests lands at about $355/mo before model costs, using TrueFoundry’s own estimator.
- Best for: regulated teams that need data residency and audit exports on paper.
- Limitation: VPC, on-prem, air-gap, and third-party guardrails are Enterprise only, and Developer allows a single budget rule.
6. Cloudflare AI Gateway: Free First Layer at the Edge
Cloudflare AI Gateway gives any Cloudflare account analytics, caching, and rate limiting for LLM calls with one line of code and no gateway fee. It suits teams whose LiteLLM proxy mostly handled caching, visibility, and simple budgets, per the Cloudflare AI Gateway docs.
Cloudflare pricing lists DLP scanning as free, guardrails billed as Workers AI inference, and a 5% fee on Unified Billing credits with provider rates passed through. Gateways created on or after September 24, 2026 follow Workers Logs pricing; older ones keep legacy limits of 100,000 logs on Workers Free and 10,000,000 per gateway on Workers Paid.
Spend limits now cap dollar cost per model, provider, or custom metadata such as user_id or agent_id, with up to 20 rules per gateway. A rule can block with a 429 or fall back to a cheaper model through Dynamic Routes, which covers the most common LiteLLM budget pattern.
- Best for: apps already on Workers or Cloudflare DNS that want a free spend and cache layer.
- Limitation: it is managed only, spend limits are eventually consistent so a burst can briefly pass a cap, and 20 rules per gateway is far simpler than LiteLLM’s key and team hierarchy.
7. Vercel AI Gateway: Zero Markup Without a Proxy
Vercel AI Gateway is a managed multi-provider gateway that charges no markup and no platform fee on tokens, including bring-your-own-key traffic. It is callable from any infrastructure, centralizes provider credentials, and logs routing attempts with ordered fallbacks across providers, per Vercel AI Gateway docs.
Budgets can cap spend for a team, a project, an API key, or a team member, per Vercel AI Gateway pricing. BYOK requires purchased credits, and when a BYOK request fails Vercel retries on system credentials and bills that fallback against your balance. Optional add-ons cost extra, such as a team-wide provider allowlist at $0.10 per 1,000 successful requests.
- Best for: teams that liked LiteLLM’s zero-fee model but no longer want to run it.
- Limitation: there is no self-hosted option, so prompts transit Vercel, and custom reporting writes bill at $0.075 per 1,000.
8. OpenRouter: Widest Catalog on One Bill
OpenRouter is a managed unified API across 500+ models from 80+ providers, and the fastest LiteLLM alternative when model breadth matters more than owning the control plane. Provider fallbacks and routing variants such as :nitro and :floor are built in, per OpenRouter.
Inference passes through at provider prices, and OpenRouter charges 5.5% with a $0.80 minimum on card credit purchases, or 5% for crypto, per the OpenRouter FAQ. BYOK includes $25,000 a month of list-price inference with no fee on pay-as-you-go, then 5% above that, per BYOK docs.
Organization guardrails add per-member and per-key USD budgets that reset daily, weekly, or monthly, plus model and provider allowlists, zero data retention enforcement, and PII redaction. Budgets are checked per user and per key, and the lower limit wins.
- Best for: startups and research teams that want hundreds of models on one key today.
- Limitation: every request leaves your network, and guardrail budgets count only OpenRouter credit spend unless you turn on BYOK inclusion.
Migration Fit: Matching Criteria to Each Alternative
Match the replacement to the LiteLLM features you actually use, then confirm each with a one-week shadow test. Teams that lean on virtual keys and budgets should shortlist Bifrost, Portkey, or TrueFoundry. Teams that mainly route and cache can move to Cloudflare, Vercel, or OpenRouter with a base URL change.
| Criterion | Bifrost | Portkey | Agent Router | Kong | TrueFoundry | Cloudflare | Vercel | OpenRouter |
|---|---|---|---|---|---|---|---|---|
| Self-host | Yes | OSS gateway | Yes | Yes | Enterprise | No | No | No |
| LiteLLM import aid | Migration tool + /litellm endpoint | Base URL swap | Config rewrite | Plugin config | Base URL swap | Base URL swap | Base URL swap | Base URL swap |
| Budgets | OSS | Paid plans | Token QuotaPolicy | Rate limits | 1 rule Dev, 10 Pro | Spend limits (20 rules) | Team, project, key, member | Guardrail budgets |
| Guardrails | Enterprise | Deterministic free; LLM and partner on Production | External service (Tetrate Enterprise) | AI plugins | Pro built-in | Llama Guard via Workers AI | ZDR routing | PII and prompt injection filters |
| Gateway fee shape | $0 OSS | Recorded logs | $0 | Per model meter | Seats + overage | $0 core | 0% markup | 5.5% on credits |
Verdict: Bifrost is the shortest path for self-hosted LiteLLM estates, Portkey is the shortest managed path, and the remaining six win when an existing platform choice such as Kong, Kubernetes, Cloudflare, or Vercel already decides the question.
A minimal shadow test before cutover
Most OpenAI-compatible gateways only need a new base URL and key, so the first test is cheap. Route 5% of staging traffic through the candidate, compare error rates, p99 latency, and spend attribution for a week, then widen the slice.
from openai import OpenAI
# Before: LiteLLM proxy
# client = OpenAI(base_url="http://litellm:4000", api_key=LITELLM_VIRTUAL_KEY)
# After: candidate gateway with an OpenAI-compatible route
client = OpenAI(base_url="http://bifrost:8080/openai", api_key=CANDIDATE_KEY)
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "ping"}],
)
print(resp.usage)- Keep budgets in parallel: mirror LiteLLM team budgets in the candidate before you shift production traffic.
- Keep traces flowing: confirm the new gateway still exports to your tracing stack, such as the tools in open source LLM observability or Langfuse alternatives.
- Pin the old proxy: leave LiteLLM pinned at a verified clean version as the rollback path until the new gateway has run a full billing cycle.
When to Stay on LiteLLM
Stay on LiteLLM when provider breadth, the MIT license, and built-in virtual keys matter more than footprint. No alternative here matches its provider list in a single self-hosted container, and its Enterprise pricing is by request capacity, never per token. A pinned, hash-verified install removes most of the supply chain exposure from March 2026.
LiteLLM is also changing. It announced an early beta Rust AI gateway on July 22, 2026, and its own vendor-run benchmark reports about 0.7 ms p99 overhead and 21.8 MB peak memory. Streaming and the full feature set were still landing at launch, so treat it as a reason to re-test in a quarter, rather than a production plan today.
What We Left Out and Why
- Helicone: it still ships an AI gateway, but Mintlify acquired Helicone on March 3, 2026 and put it in maintenance mode. See Arize Phoenix vs Langfuse vs Helicone for the observability angle.
- GoModel and similar young Go gateways: they post strong footprint numbers, and we will review them once they publish governance features and production adopters.
- Pure observability suites: tools that trace calls without routing them are covered in AI agent observability tools, LangSmith alternatives, and Langfuse vs LangSmith vs Braintrust.
Our Recommendation for Production LLM Routing
For a self-hosted LiteLLM estate with virtual keys and budgets, start a Bifrost shadow test this sprint; for a team that wants out of gateway operations, trial Portkey Production. Choose Agent Router, Kong, or TrueFoundry when Kubernetes, Kong, or a private contract already sets the rules.
Pick Cloudflare, Vercel, or OpenRouter when the gateway’s main job is routing, caching, and one bill.
Questions about a gateway we missed or a price that changed can go to our contact page. Teams pairing a gateway with eval tooling should read AI agent evaluation tools and Braintrust vs Maxim vs Galileo, since Bifrost and Maxim share a vendor.
Related Guides
- AgentOps hub
- AI Gateways category
- AgentOps tools map
- Best AI agent observability tools
- Open source LLM observability
- Langfuse alternatives
- LangSmith alternatives
- Langfuse vs LangSmith vs Braintrust
- Arize Phoenix vs Langfuse vs Helicone
- Braintrust vs Maxim vs Galileo
- AI agent evaluation tools
- LangSmith vs Langfuse vs Braintrust pricing
- ToFu MCP tool security bugs
- AI agent frameworks
- Editorial standards · Disclosure · Contact
Frequently Asked Questions
Is LiteLLM still safe to use after the March 2026 supply chain attack?
Yes, if you pin a verified version. Only PyPI releases 1.82.7 and 1.82.8 were malicious, and LiteLLM verified 1.82.6 and earlier as clean. Pin versions, check hashes, and rotate every credential on any host that installed the bad releases, because version 1.82.8 ran on every Python startup.
What is the easiest LiteLLM alternative to migrate to?
Bifrost is the easiest self-hosted move, because its migration tool copies models, keys, teams, users, and virtual keys from a running LiteLLM proxy without changing it. Managed options such as Portkey, Vercel, and OpenRouter usually need only a new base URL and key, but budgets must be rebuilt by hand.
Which LiteLLM alternatives can be self-hosted for free?
Bifrost and Agent Router are free to self-host under Apache 2.0, and Portkey’s open-source gateway is free to run locally. Kong Gateway has an open-source core, though several AI plugins and the Konnect control plane are paid. TrueFoundry supports VPC and air-gapped installs only on its Enterprise contract.
Do LiteLLM alternatives charge a markup on model tokens?
Most do not. Vercel states 0% token markup including BYOK, Cloudflare passes provider rates through and charges 5% only on Unified Billing credits, and OpenRouter charges 5.5% on card credit purchases. Portkey bills recorded logs, TrueFoundry bills seats and requests, and Kong meters distinct models proxied.
Should I wait for the LiteLLM Rust gateway instead of switching?
Wait only if footprint is your single complaint and you can tolerate a beta. LiteLLM announced the Rust gateway on July 22, 2026 with about 0.7 ms p99 overhead in its own benchmark, while streaming and full features were still landing. Teams with security or license concerns should test an alternative now.