Portkey
FreeSelf-host / Cloud
FreeFree tier
Inputs: text, code, image, audioOutputs: text, code, image, audio
About Portkey
Portkey AI Gateway is a blazing fast, open-source AI gateway that provides a unified API to route requests to over 1,600 language, vision, audio, and image models from 40+ providers. It features integrated guardrails, automatic retries and fallbacks, load balancing, and conditional routing for enterprise-grade reliability. With a tiny footprint (122kb) and 1ms latency, it processes over 10 billion tokens daily. The gateway can be self-hosted or used via Portkey Cloud, and integrates seamlessly with popular frameworks like LangChain, LlamaIndex, and OpenAI SDKs. It also includes a Gateway Console for local logging and supports agentic workflow integrations and MCP server management.
Key Features
Unified API to 1,600+ language, vision, audio, and image models across 40+ providers
Blazing fast (1ms latency) with a tiny footprint (122kb)
Battle-tested, processing over 10 billion tokens daily
Enterprise-ready with enhanced security, scalability, and custom deployments
Integrated guardrails for AI safety
Automatic retries and fallbacks to prevent downtime
Load balancing and conditional routing for scaling AI apps
Multi-modal capabilities beyond text
Agentic workflow integrations and MCP server management
Gateway Console for local logging and observability
Pros & Cons
Pros
- Open-source and free to self-host
- Exceptionally low latency (1ms) and small footprint (122kb)
- Supports a vast range of models and providers (1,600+ models)
- Includes built-in guardrails for safety and reliability
- Automatic retries, fallbacks, and load balancing for high availability
- Easy integration with popular AI frameworks and SDKs
- Active community and enterprise support options
- Multi-modal support (text, vision, audio, image)
Cons
- Requires Node.js and npm for local self-hosting
- Cloud version may have additional costs not specified in open-source repo
- Documentation and setup may be less intuitive for non-developers
- Some advanced features (e.g., enterprise auth) may require cloud subscription
Best For
Integrating any LLM into applications with a single APIBuilding resilient AI applications with automatic failover and retriesScaling AI workloads with load balancing and routing policiesDeploying safety guardrails on AI model interactionsDeveloping multi-modal applications (vision, audio, image)Implementing agentic workflows and managing MCP serversSelf-hosting an AI gateway for enterprise compliance and control
FAQ
What models can I access with Portkey AI Gateway?
Over 1,600 language, vision, audio, and image models from 40+ providers including OpenAI, Anthropic, Bedrock, Groq, and more.
Is Portkey AI Gateway open-source?
Yes, it is open-source and available on GitHub under the Portkey-AI organization. You can self-host it or use Portkey Cloud.
How fast is the gateway?
The gateway has a latency of 1ms and a tiny footprint of 122kb, making it extremely fast and lightweight.
Does the gateway support multi-modal models?
Yes, it supports language, vision, audio, and image models beyond just text.
How do I get started?
Run 'npx @portkey-ai/gateway' locally (requires Node.js and npm) to start the gateway on localhost:8787. Follow the quickstart guide in the README.