What is the best AI gateway in 2026?

LiteLLM is the default open source answer to "how do I put every model behind one key?". The LiteLLM AI Gateway is a self-hosted server: your apps call one OpenAI-compatible endpoint, and it forwards each request to Anthropic, OpenAI, Bedrock, Vertex, Mistral or any of the 140+ providers it integrates.

The value is in the keys. Each team or project gets a virtual key, and each key has its own models, budgets and rate limits. Budgets are hard: when a team hits its cap, requests stop. The gateway records the cost of every request, so finance gets spend per key, team, user and tag without a spreadsheet. Load balancing, fallbacks and retries move traffic when a provider fails.

The open source version costs $0. The Enterprise tier adds JWT auth, SSO, audit logs and support with custom SLAs.

The catch is that you run it. You own the servers, the database, the upgrades and the on-call pager. A team that compared five gateways on Reddit in 2026 found it great for getting started, but said per-user rate limits are painful to configure and RBAC feels bolted on at enterprise scale. And in 2026 two PyPI releases (1.82.7 and 1.82.8) were compromised for about 40 minutes; the official Docker image was not affected.

Tip: deploy the official Docker image and pin the version tag. Never run an unpinned pip install litellm in a build pipeline.

OpenRouter is a hosted marketplace: one API key, one credit balance, 600+ models from 80+ providers. Swap a model by changing a string. When a provider is down, its routing retries another one automatically.

Pricing is simple once you read it right. Inference is billed at the provider's list price on every plan. OpenRouter's fee is charged when you buy credits: 5.5% on Standard and 8% on Business, never on individual requests. So $1,000 of credits costs $1,055 on Standard. Bring your own provider keys (BYOK) and the first $25,000 of list-price inference each month carries no fee, then 5%.

Free models exist, capped at 20 requests a minute and 50 a day, rising to 1,000 a day once your account has bought at least $10 of credits. In a 2026 Reddit thread, one developer balks at the 5.5% fee and looks at LiteLLM instead, while another team says reliability and fallbacks matter more to them than the fee. A detail in OpenRouter's favor: an attempt that errors is not charged for its model tokens, so with fallback you pay only for the run that answers.

The limit: your prompts go through a third party. SSO and managed policy enforcement are Enterprise-only.

Tip: if your spend is mostly on two providers you already pay, use BYOK. Under $25,000 a month, you keep OpenRouter's routing and fallbacks without the fee.

Vercel AI Gateway gives you 400+ models behind one key with automatic fallbacks during provider outages. It plugs straight into Vercel's AI SDK, and the docs cover use from any infrastructure.

The pricing is the draw. On the paid tier, the gateway provides tokens with zero markup, including when you bring your own key. Every Vercel team also gets $5 of credit a month on the free tier, limited to a subset of models with lower rate limits. You buy credits as you go, with no commitment.

Zero markup does not mean zero cost. Some controls are paid add-ons: team-wide zero data retention and a team-wide provider allowlist each cost $0.10 per 1,000 requests, and AI Gateway traces are billed per 1,000. Check the add-on list before you turn everything on.

The limit: it is hosted only, so your traffic goes through Vercel. BYOK is not available on the free tier, and on the paid tier a request that fails with your key is retried with Vercel's credentials and charged to your credits; AI Gateway budgets also exclude BYOK spend, so set spend limits at the provider too.

Tip: claim your $5 of free credit to prototype, then buy credits only when you move to production. Credits carry no commitment.

Portkey started as an open source gateway (MIT licensed, a small footprint) and grew into a control plane: routing to 1,600+ LLMs, fallbacks, automatic retries, load balancing, guardrails, prompt management and full request traces.

Pricing is by recorded logs, not tokens. The Developer plan is free with 10k recorded logs a month and 3-day log retention, and Portkey itself says it is not suitable for production workloads. Going over the limit does not block requests; only the extra logs are not recorded. Production is $49 a month with 100k logs, then $9 per extra 100k. Enterprise is custom.

The big change: in 2026 Palo Alto Networks completed its acquisition of Portkey and made it the gateway of its Prisma AIRS security platform. Good news for security teams that buy Palo Alto; a question mark for small teams on the cheap plans.

The limit: a team that compared gateways on Reddit in 2026 could not get the VPC deployment to work as smoothly as advertised. Self-hosting the open source gateway is free, but logs live in Portkey's console, where the free plan keeps them 3 days and Production 30 days.

Tip: start with the open source gateway in front of your app, then send logs to the hosted console only for the routes you need to audit.

Cloudflare AI Gateway sits on Cloudflare's network in front of your provider calls. It takes one line of code: change the base URL. You get analytics, logging, caching, rate limiting, request retries and model fallback.

The core features are free on every plan. On accounts with the older log system, the Workers Free plan keeps 100,000 logs in total across all gateways, and Workers Paid keeps 10 million per gateway; accounts that created their first gateway later in 2026 store logs in Workers Logs, under its own pricing and retention. If you want one bill instead of your own provider keys, Unified Billing passes token prices through with no markup but adds a 5% fee on credits: $100 of credit costs $105.

The caching is the money saver. Identical prompts (an FAQ bot, a classification job) are served from Cloudflare's cache instead of the provider.

The limit: governance stays basic. For budgets per team and audit trails, pair it with Portkey or LiteLLM. A 2026 Hacker News thread on Cloudflare's AI platform also notes that zero data retention is not on by default and not supported by every provider.

Tip: turn caching on per route, not globally. Cache a support FAQ; never cache a prompt that carries user data.

TrueFoundry sells an AI gateway that you deploy in your own VPC, on-prem or air-gapped. It routes to 2,500+ models, adds rate limits, budgets, RBAC, guardrails and logs, and also manages MCP servers and agents.

The pricing page is clear. Developer is $0 for up to 3 users, with 10K requests and tool calls per user a month and 7-day log retention. Pro is $25 per user a month with 20K requests per user, pooled across the team, then $20 per extra 100K. Model-provider costs are pass-through: TrueFoundry does not mark up tokens. TrueFoundry's own estimator puts 7 users and 1 million requests at $355 a month.

A team that compared five gateways on Reddit in 2026 picked it for VPC deployment and fallback chains that work (the post reads partly like a pitch, so weigh it lightly).

The limit: it is a platform, not a library. For a solo developer, LiteLLM or OpenRouter is faster to start.

Tip: count requests and tool calls together, since each MCP tool call counts as one request. Agent loops burn the per-user quota faster than chat, and active users are counted at the month's peak, so remove departed users before the month starts.

Bifrost, from Maxim AI, is an open source gateway under the Apache 2.0 license. You start it with one npx command and get an OpenAI-compatible API, failover, load balancing, semantic caching, virtual keys with budgets, and an MCP gateway.

Its pitch is speed. In the vendor's own benchmark, Bifrost adds 11 microseconds of overhead per request at 5,000 requests per second on a t3.xlarge machine. That is a vendor number, not an independent one. It is still why teams look at Bifrost when a Python proxy starts to lag under load.

It costs nothing to self-host. Enterprise features come with a 14-day trial, then a conversation with sales.

The limit: it is younger than LiteLLM, with a smaller community, and the benchmarks that put it ahead of LiteLLM ("50x faster" at P99 latency) are published by Maxim AI itself.

Tip: Bifrost ships a LiteLLM compatibility mode, so you can move an existing LiteLLM setup without rewriting your client calls.

Kong AI Gateway is a set of AI plugins on top of Kong's API gateway: AI Proxy to reach several providers through one interface, AI Rate Limiting Advanced, AI Semantic Cache, AI Semantic Prompt Guard and AI Sanitizer. The control plane runs in Kong Konnect, serverless or hybrid.

Pricing is the weak spot for small teams. Konnect Plus starts at $25 a month plus usage, billed monthly, with a 30-day free trial. A serverless AI gateway control plane is $25 a month, and every unique LLM proxied by the AI plugins costs $100 a month. Five models means $500 a month in model fees, or $525 with the control plane, before traffic.

It shines where Kong already runs: the same team, same policies, same dashboards for APIs and LLMs. The same 2026 Reddit comparison found the learning curve steep and said it felt like AI features retrofitted onto an API gateway.

Tip: count your models before you sign. Route rarely used models through one generic proxy route rather than one per model.