Searches for LiteLLM alternatives climbed in March 2026, after attackers compromised the security scanner in LiteLLM's build pipeline, took its publishing credentials, and pushed two backdoored releases to PyPI. Two more vulnerabilities in the proxy's own code followed, and all three reached the Cybersecurity and Infrastructure Security Agency (CISA) Known Exploited Vulnerabilities catalog between March and June. Most of the searching that followed assumes self-hosting the proxy was the mistake.
None of that settles whether to keep running the proxy yourself. What it reopens is custody. A self-hosted proxy keeps provider keys and prompts inside your network and hands you the patching, and a managed gateway reverses both.
LiteLLM deploys as a server on Vercel and routes through AI Gateway behind a vercel_ai_gateway/ prefix. The seven options below are compared on license, hosting model, provider coverage, operational footprint, and patch record.
Key takeaways:
Self-hosting costs throughput, added latency, and three stateful services on the on-call rotation, which is the price of keeping keys and prompts inside your network.
Provider counts do not compare across vendors, and LiteLLM's own figures range from 100+ providers in its docs to 140+ providers and 1,800+ models on its site.
Two of the seven options here changed owners inside twelve months and one moved to maintenance mode, which makes continuity a sharper filter than feature parity.
A gateway is the highest-value target in an AI stack, because it holds every provider key and reads every prompt in plaintext.
The migration cost sits in accumulated observability data, caching rules, and routing configuration, not in the
base_urlswap.
Copy link to headingWhat LiteLLM's 2026 security record changes
LiteLLM is the most-adopted option here, with 57,000 GitHub stars, production deployments at NVIDIA, Netflix, and Okta, and broader provider coverage than anything else in this comparison, all under an MIT license. Its security year matters to so many teams because of that reach.
Those three listings came from two different kinds of failure. One was an attack on the build pipeline that never touched LiteLLM's code, and the other was a pair of vulnerabilities in the code itself.
Copy link to headingThe March supply-chain compromise
LiteLLM's continuous integration pipeline ran Trivy, a security scanner that attackers had compromised between March 19 and 23, 2026. Credentials taken from that pipeline were used to publish two poisoned releases straight to PyPI on March 24, bypassing the project's own release workflow. Versions 1.82.7 and 1.82.8 were live from 10:39 UTC for roughly 40 minutes before PyPI quarantined them. The payload harvested environment variables, SSH keys, cloud credentials, and database passwords from any environment that installed them.
Teams running the official Docker image were unaffected, because that path pins its dependencies in requirements.txt. LiteLLM's response was substantive: paused releases pending a supply-chain review, SHA-256 checksums published for audited releases, and v1.83.0 shipped through a rebuilt pipeline with isolated environments. The DSPy maintainers pinned the dependency at 1.82.6 or lower while they reconsidered it.
Copy link to headingTwo proxy vulnerabilities exploited in the wild
The supply chain was not the only problem. Two flaws in the proxy's own code reached the KEV catalog within 60 days of each other, both scoring above 8.5 on the Common Vulnerability Scoring System (CVSS):
CVE-2026-42208, SQL injection, CVSS 9.3: An unauthenticated request could read or modify the proxy's database, because the API key check built its query from the caller's own key. A crafted
Authorizationheader on any model route was enough. Attacks appeared 36 hours after the advisory, and the KEV listing followed on May 8, 2026.CVE-2026-42271, command injection, CVSS 8.7: An authenticated user could run arbitrary commands on the host through two Model Context Protocol (MCP) preview endpoints, which accepted
command,args, andenvvalues without restriction. The KEV listing followed on June 8, 2026.
Both are fixed in v1.83.7-stable and later, and a third advisory chains with them toward remote code execution. Severity ratings under-read in this category, because the component holding every provider key also reads every prompt in plaintext, which means a database read is a credential dump. Who absorbs that patching load is the first thing to settle.
Copy link to headingEvery LiteLLM alternative is a self-hosted proxy or a managed gateway
Deployment architecture decides more than product selection does, so it comes first. Self-hosting buys custody. Prompts and provider keys never leave your network, the license cost is zero, and nothing about the request path is opaque to you. The costs land in three places, and they compound as traffic grows rather than showing up on day one:
Throughput: A team running v1.80.15 reported throughput dropping from roughly 16 to 9 requests per second at 500 concurrent requests, and neither disabling spend logs nor adding pgbouncer recovered it. The Python global interpreter lock sets a ceiling that configuration cannot move, which is why a Rust rewrite of the
/chat/completionspath is underway.Latency: A proxy running in one region adds a full network round-trip in each direction for every request that originates elsewhere. No amount of tuning removes that, and published overhead figures come from the vendors themselves, so they are worth little in a comparison.
Operational footprint: A production deployment is the proxy, PostgreSQL, and Redis, which puts three stateful services on the on-call rotation before any model call happens.
None of that settles the question, because the two models are not exclusive. A self-hosted proxy can front a managed gateway, which keeps the local control plane and hands off provider fan-out:
- model_name: gpt-5.4-gateway litellm_params: model: vercel_ai_gateway/openai/gpt-5.4 api_key: os.environ/VERCEL_AI_GATEWAY_API_KEYA migration that only has to move provider fan-out is a far smaller project than one replacing the control plane, and the two routinely get scoped as though they were the same job. With that split in hand, the products themselves become comparable.
Copy link to headingLiteLLM alternatives compared on license, hosting, and maturity
With the architecture question settled, the field narrows to what each option costs to license, run, and keep:
Five of the seven can run inside your own network, and the two that cannot are the only ones with no infrastructure to operate. Those five come first, since they carry the same patching obligation LiteLLM does.
Copy link to headingSelf-hosted alternatives to LiteLLM
Kong, Portkey, Helicone, and Envoy AI Gateway all leave the patch cadence with you, exactly as LiteLLM does, so the record that prompted the search applies to them too. Runtime and surrounding infrastructure are where they diverge.
Copy link to headingEnvoy AI Gateway
Envoy AI Gateway is a Kubernetes-native gateway with a Go control plane driving Envoy's C++ data plane through custom resource definitions (CRDs). Its first stable release landed in June 2026, which makes it the youngest option here by a wide margin. The Apache 2.0 core ships AWS IAM, Azure Service Principal, and GCP Workload Identity as first-class provider credentials, so cloud-native authentication does not sit behind a commercial tier.
The migration cost is the operating model. Teams already operating Kubernetes alongside Envoy or a service mesh extend the infrastructure they patch anyway. Everyone else absorbs a cluster and a CRD-driven configuration model to route model traffic.
Copy link to headingLiteLLM vs Envoy AI Gateway
LiteLLM and Envoy AI Gateway are compared more than any other pair in this field, and the differences are structural. Configuration is the first split. LiteLLM is a Python application driven by a YAML file, which one engineer can stand up and read. Envoy AI Gateway expresses the same intent as Kubernetes custom resources, which is heavier to start and considerably better at surviving a team change.
Coverage runs the other direction, with 16 providers at Envoy's v1.0 release against LiteLLM's 100+. Identity splits again, since Envoy puts cloud-provider credentials in its open-source core while LiteLLM keeps single sign-on (SSO) and OpenID Connect (OIDC) in its enterprise tier.
The operational floor is what usually decides it. LiteLLM needs three stateful services and Envoy AI Gateway needs a current Kubernetes cluster. Whichever of those you already run is the cheaper answer, and neither is worth adopting for the gateway alone.
Copy link to headingKong AI Gateway
Kong layers AI plugins onto the Apache 2.0 Kong Gateway core, where they arrived in Kong Gateway 3.6. The 3.x line is generally available, the AI Gateway 2.0, announced in July 2026, remains in private beta, and the advanced AI plugins and LLM analytics sit in the Enterprise and Konnect tiers.
Where Kong already handles API governance, the plugin model extends a system the team maintains and patches, which is the cleanest adoption path in this group. Kong has not named third-party AI gateway adopters, though, so the evidence base for a production migration is thin. For anyone else, adopting a full API gateway to route model traffic is a large step for a narrow gain.
Copy link to headingPortkey
Portkey's open-source gateway is MIT-licensed and carries more routing machinery than anything else here, covering fallbacks and conditional routing, canary testing, circuit-breaker retries, and 50+ guardrails. For a team leaving LiteLLM specifically for policy enforcement, this is the closest match in the self-hosted field.
Ownership is the open question. Palo Alto Networks closed its acquisition of Portkey on May 29, 2026, and Portkey is now the AI Gateway inside Prisma AIRS, which reached general availability in July. The self-hosted v2.0 is labeled Pre-Release, and the open-source repository's last push predates the close.
Both things are true at once. The routing capability is real and available today under MIT. The roadmap for the self-hosted path now runs through an enterprise security platform, and the trillions-of-tokens scale figures the company cites describe the hosted service.
Copy link to headingHelicone
Helicone's gateway is a single Rust binary, and it is the lightest thing to operate in this group. The license needs care, because the gateway repository ships under GPL-3.0 even though the broader Helicone platform is Apache 2.0.
Mintlify acquired Helicone on March 3, 2026. The stated position is maintenance mode, covering security updates, bug fixes, and new model support with no new feature work, and Mintlify has said it is helping teams migrate to another platform. The gateway repository has not been pushed to since November 2025, and its routing configuration documents fall back without conditional routing or a retry-with-backoff policy.
That makes it a lateral move rather than a migration. Helicone still works well as a stable router with logging attached, which is a reasonable thing to want and a poor thing to build a two-year roadmap on.
Copy link to headingManaged gateways as LiteLLM alternatives
Vercel AI Gateway and OpenRouter both put a vendor on call for patching and scaling, which resolves the concern that sent most teams searching. What you trade for it is the option to run the gateway on hardware you control.
Copy link to headingOpenRouter
OpenRouter is a proprietary managed service covering 70+ providers, and it has the broadest model catalog in this comparison. Free model variants carry a daily request cap that rises once an account has purchased $10 in lifetime credits.
Every request traverses OpenRouter's layer, which is the deciding factor for anyone with a data residency requirement. The reliability record includes a database outage in August 2025 and two February 2026 outages traced to caching dependency failures.
For prototyping with zero operational overhead, OpenRouter fits better than anything else here. Developers who keep it past the prototype stage tend to hold a direct-provider path in reserve for the outage windows.
Copy link to headingVercel AI Gateway
AI Gateway routes to hundreds of models through a single endpoint at ai-gateway.vercel.sh/v1, with no markup on tokens, including when you bring your own key. The free tier includes $5 a month in credits against a subset of models with lower rate limits, and the paid tier adds the full catalog, BYOK, and custom rate limits.
The routing layer is where the operational difference concentrates. Requests run across 126 points of presence and 20 compute regions rather than one, and automatic cross-provider fallback rescued 3.5% of requests, 5.1% of tokens, and 4.9% of cost through April 2026. Cline A/B tested its previous router against AI Gateway on live traffic for a week. P99 streaming latency improved 10 to 14% and API error rates dropped 43.8%, at under 20ms of added overhead.
OIDC tokens are provisioned automatically for applications deployed on Vercel, so there are no long-lived gateway keys to rotate. Routing Rules rewrite model requests team-wide without a deploy, which covers a provider outage or a model swap without touching application code. Zero Data Retention routing is available per request and team-wide. All of it assumes a managed service, so a hard requirement to keep prompts and provider keys on your own infrastructure points back to the five self-hosted options.
Copy link to headingWhich LiteLLM alternative fits your infrastructure
The field sorts cleanly once the choice is framed around infrastructure you already operate, rather than features you might use:
Strict data residency, air-gapped, or regulated environments: Self-hosted LiteLLM, because prompts and keys never cross your network boundary, which is the same reasoning that pushes teams toward self-hosted models in the first place.
Existing Kubernetes with Envoy or a service mesh: Envoy AI Gateway, which extends a control plane the team already runs and patches.
Existing Kong API governance: Kong AI Gateway, through a plugin model the team maintains already.
Guardrails and policy enforcement as the primary need: Portkey, with the roadmap question under new ownership priced in.
Broadest model access for rapid prototyping: OpenRouter, with a direct-provider fallback kept warm.
Already deploying on Vercel: AI Gateway, with no database or cache to operate alongside it, since it runs on the same platform as the deployments calling it.
These gateways generally expose OpenAI-compatible APIs, so the mechanical part of a migration is changing a base_url. The expensive part is everything that accumulated around the old gateway, including observability history that stops at the cutover and routing behavior that encodes decisions nobody wrote down.
Copy link to headingChoose for the team you have
Grabbing whichever alternative tops a list is the instinct after an incident. The better question is who patches the gateway next month, because a self-hosted proxy is a security-critical service that holds every provider key and reads every prompt in plaintext. Three exploited vulnerabilities in four months is the cadence that the job now runs at.
A team that can staff that work should self-host, and keep the keys and prompts inside its own network at zero license cost. A team that cannot should hand the patching to a vendor, and accept that the traffic leaves the network to get there. Most teams running LiteLLM today inherited it from whoever prototyped with it, and it earned that position on coverage and configurability that the rest of this field has not matched. The honest answer more often turns on headcount than on features.
For teams choosing the managed path, AI Gateway covers the operational layer:
Automatic cross-provider failover: Fallback routing rescued 3.5% of requests and 5.1% of tokens through April 2026, with no retry logic in application code.
OIDC credential injection: Short-lived tokens replace the static gateway key, so there is no production secret waiting on a rotation schedule.
Routing Rules: Model routing is rewritten team-wide from the dashboard, with no code change and no redeploy.
Zero-markup tokens with BYOK: You pay provider list rates, including on requests that carry your own keys.
Zero Data Retention routing: Requests route through providers under no-retention agreements, per request or team-wide, for workloads that cannot store prompt or completion data.
Start a new project and route your first model call through AI Gateway, or browse vercel.com/templates for AI applications already wired for production.
Copy link to headingFrequently asked questions about LiteLLM alternatives
Copy link to headingWhich open source AI gateways are production-ready in 2026?
LiteLLM at v1.98.0 and Envoy AI Gateway at v1.1.0 are the two with stable releases and active development. Kong Gateway 3.x is generally available, while Kong AI Gateway 2.0 is in private beta, Portkey's self-hosted v2.0 is Pre-Release, and Helicone's gateway is maintenance-only.
Copy link to headingWhich services provide a single endpoint for multiple AI model providers?
Six of the seven options here do: LiteLLM, Portkey, Envoy AI Gateway, Helicone, OpenRouter, and Vercel AI Gateway each expose one OpenAI-compatible endpoint that routes to multiple providers. OpenRouter covers 70+ providers, AI Gateway reaches hundreds of models through ai-gateway.vercel.sh/v1, and LiteLLM documents 100+ providers.
Copy link to headingWhich AI gateways offer a free tier or open-source version?
LiteLLM, Envoy AI Gateway, Kong, Portkey, and Helicone are open source and self-hostable, so you pay only for infrastructure and provider tokens. OpenRouter caps daily requests on free model variants, and AI Gateway includes $5 a month in credits at zero markup.
Copy link to headingWhat does it cost to migrate off LiteLLM?
Less than teams expect at the API layer and more than they expect everywhere else. Every option here speaks the OpenAI format, so the application code changes by one base_url. Spend history, cache keys, and per-team routing rules do not transfer, and rebuilding them is the actual project.