About Cloudflare AI Gateway
Cloudflare AI Gateway provides observability and control for AI applications. By connecting your apps to the gateway, you gain insights into usage through analytics and logging, and can control scaling with caching, rate limiting, request retries, and model fallback. It supports major AI providers including OpenAI, Anthropic, Google Gemini, Replicate, and Workers AI, and integrates with a single line of code. The service runs on Cloudflare's global edge network and is available on all Cloudflare plans.
Key Features
Analytics and logging to monitor requests, tokens, and costs
Caching of responses from Cloudflare's edge to reduce latency and costs
Rate limiting to control how your application scales
Request retry and model fallback for improved resilience
Support for multiple AI providers (OpenAI, Anthropic, Google Gemini, Replicate, Workers AI, and more)
One line of code integration with existing applications
Pros & Cons
Pros
- Easy integration with just one line of code
- Runs on Cloudflare's global edge network for low latency
- Supports a wide range of popular AI providers
- Caching reduces both latency and API costs
- Rate limiting helps prevent overuse and manage scaling
- Retry and fallback features increase application resilience
Cons
- Requires a Cloudflare account and usage of Cloudflare's infrastructure
- Limited to the set of integrated AI providers (may not cover all custom or niche models)
Best For
Gain insights on how users interact with your AI applicationControl API costs and prevent abuse with caching and rate limitingImprove reliability of AI applications through automatic retries and fallback modelsMonitor and log errors for debugging and optimization
FAQ
What is Cloudflare AI Gateway?
AI Gateway is a service that provides observability and control over AI applications, offering analytics, logging, caching, rate limiting, and request retries/fallback.
How do I get started with AI Gateway?
You can get started by following the Get started guide on the Cloudflare developer docs. It only requires one line of code to configure your application with AI Gateway.
Which AI providers are supported by AI Gateway?
AI Gateway supports Workers AI, Anthropic, Google Gemini, OpenAI, Replicate, and more providers.
Does AI Gateway support caching?
Yes, AI Gateway can serve requests directly from Cloudflare's cache instead of the original model provider, reducing latency and cost.
Can I control how my application scales with AI Gateway?
Yes, AI Gateway includes rate limiting features to control the number of requests your application receives.