// the find
Portkey-AI/gateway
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
Portkey Gateway is a Node.js proxy that sits in front of your LLM calls and gives you a single OpenAI-compatible endpoint for 250+ models. It handles retries, fallbacks, load balancing, and output guardrails through a declarative config object. Teams building multi-provider AI apps who want resilience logic without baking it into application code are the target.
The guardrail plugin architecture is genuinely well done — 20+ third-party providers (Azure Content Safety, Bedrock, Pangea, etc.) all conform to the same interface, so swapping or chaining them is mechanical. The routing config is declarative and composable: you can express 'try GPT-4o, fall back to Claude on 429s, retry 5 times with exponential backoff' in a small JSON blob rather than application code. The 122kb bundle that runs on Cloudflare Workers is real — edge deployment without a Node server is actually usable here. OpenAI-compatible API surface means it's a one-line change to point existing SDKs at it.
The open-source version is deliberately hobbled: semantic caching, observability/tracing, prompt template management, and provider cost optimization all require the hosted cloud product. The README lists these features prominently but buries the asterisk. The guardrail plugins defeat the latency pitch — if your output guardrail makes a synchronous HTTP call to a third-party API, the '<1ms gateway' claim is irrelevant; you're now waiting on Pangea or Azure. Gateway 2.0 has been in 'pre-release' since before the last push date, which is a mild yellow flag about the project's release discipline. Self-hosting the full feature set isn't really possible without paying.