LangChain has opened the public beta of the LangSmith LLM Gateway, a central governance layer designed to give teams runtime control over every LLM call their production agents make. The announcement, published on July 30, 2026, positions the gateway as a way to enforce spend caps, rate limits, model fallbacks, and sensitive data redaction from a single point, rather than scattering those controls throughout agent code.
The product sits between agents and the models they call, acting as a policy enforcement point. It is available now for Plus and Enterprise plan users, and it integrates directly with LangSmith, the company's agent engineering platform, logging gateway events as metadata in traces.
The Problem: Uncontrolled Agents Are Expensive
The core argument from LangChain is that agents in production need runtime controls to avoid cost overruns and outages. Without proper tooling, engineers have to write controls manually within each agent, which leads to multiple implementation paths and inconsistent enforcement. The model layer, the company says, can become a source of costly mistakes and customer-facing outages.
Two concrete examples illustrate the stakes. A provider outage can cause customer service agent errors, leading to customer frustration and lost revenue. More dramatically, a coding agent stuck in a retry loop overnight can generate 10,000 LLM calls, resulting in a four-figure invoice. That scenario, the company argues, shows why it is not enough to understand what happened after the fact; teams need to enforce policies before violations occur.
The gateway addresses this by letting teams define policies once and enforce them consistently across every call. A model gateway, the company says, also helps avoid vendor lock-in, because teams can route traffic to different providers without rewriting agent logic.
Spend Caps and Rate Limits at Four Levels
The gateway's cost controls are granular. Spend caps can be set at four levels: organization, workspace, API key, and user. When a spend cap is hit, the agent receives a 402 response with a clear error, so the failure is explicit rather than silent.
Rate limits follow the same pattern, applied at organization, workspace, user, and API key levels. This allows teams to protect shared infrastructure from a single runaway workload.
For multi-customer deployments, the gateway supports per-customer policies through a custom request header that identifies end customers or teams. This means all LLM calls can be routed through one API key while enforcing separate spend caps and rate limits per customer. No customer-specific API keys are required, which simplifies billing and access management.
Fallback rules can also be defined across models and hosts. If a primary provider fails or returns an error, the gateway can route the request to a secondary model or host automatically, reducing the chance of a customer-facing outage.
Data Protection and Provider Support
The gateway detects, redacts, and replaces personally identifiable information (PII) and secrets in requests before they reach model providers. Sensitive data is also redacted from traces recorded in LangSmith, so internal logs do not leak customer information. The Data Protection redaction feature is currently available only to Enterprise plan users.
Provider support includes OpenAI, Anthropic, Fireworks, and custom models with OpenAI- or Anthropic-compatible endpoints. The gateway is BYOK-first, meaning users can bring their own provider keys. For teams that prefer hosted inference, the gateway also supports open models powered by Fireworks, available through LangSmith using Gateway Credits.
Lin Qiao, CEO of Fireworks, endorsed the partnership. "Open weight models give teams more control over their AI systems, and we're excited to make them easier to adopt in production through the LangSmith Gateway," she said. Fireworks provides fast, efficient inference for open models, and the company argues that open models are already delivering the performance needed for production AI.
Early Users Report Centralized Billing and Hard Caps
Stay ahead of the AI curve
The most important updates, news, and content — delivered weekly.
No spam. Unsubscribe anytime.
Two customers shared their experience with the gateway. Sean Rich, cofounder and CTO of Blueberry AI, said the gateway helped his team centralize billing and cost tracking for Deep Agents. Hylke Sybesma, AI Engineer at Friday Digital Agency, emphasized the importance of hard limits. "Every production agent needs a hard spend cap, and the gateway makes sure our costs never exceed those limits," he said.
Both quotes point to a common theme: teams are already using the gateway for public-facing agents with spend caps and for multi-customer agents with centralized billing. The gateway can be called directly via API or used with coding harnesses like Claude Code, Codex, and Deep Agents Code (dcode).
Getting started is straightforward. Teams point their agents at the LangSmith Gateway endpoint, authenticate with a LangSmith API key, add provider API keys to workspace secrets, configure controls, and update the base URL. To try it, users can log into LangSmith or sign up, then select "LLM Gateway" in the sidebar. Enterprise plan users can request access to Data Protection controls.
Roadmap: Guardrails and CI/CD Controls
The announcement also outlines future plans. LangChain says broader guardrails and CI/CD controls are on the way, including deploying prompts and context, A/B tests, blue-green and shadow deployments, and Infrastructure-as-Code support. These additions would extend the gateway from runtime enforcement into the deployment pipeline.
For now, the public beta focuses on the core runtime controls. The blog post, written by Martha Janicki and Ramon Petgrave, runs about 7 minutes in reading time and frames the gateway as a response to a specific pain point: agents in production need guardrails, and those guardrails need to live in one place.
The company's position is that controls embedded in each agent's code lead to multiple implementation paths, which are hard to audit and easy to break. A model gateway, by contrast, lets teams define policies once and enforce consistently, whether the agent is a simple chatbot or a complex multi-step workflow.
The timing is notable. As agents move from demos to production, the failure modes shift from "does it work" to "how much does it cost and can it stay up." The gateway is LangChain's answer to those operational questions, and the early customer quotes suggest the approach is resonating with teams that have already felt the pain of uncontrolled spend.
Whether the gateway will become a standard part of the agent stack remains to be seen, but the beta launch gives teams a concrete tool to test. With spend caps at four levels, rate limits at four levels, and fallback rules across models and hosts, the gateway covers the most common operational controls in one package.
The 10,000-call retry loop example is a useful benchmark for teams evaluating the product. If a single overnight incident can generate that many calls and a four-figure invoice, the cost of a governance layer is easy to justify. The gateway's 402 response on spend cap hits ensures that when the limit is reached, the system fails loudly and clearly, not silently and expensively.
Data redaction adds another layer of safety, particularly for enterprises handling customer data. By redacting PII and secrets before they reach model providers, and again from traces, the gateway reduces the risk of data leakage through logs or third-party APIs.
The BYOK-first approach is also worth noting. Teams that already have contracts with OpenAI, Anthropic, or Fireworks can keep those relationships and simply add the gateway as a policy layer on top. For teams that want hosted open models, Gateway Credits offer a path to pay for inference through LangSmith without a separate Fireworks account.
The future roadmap items, including CI/CD controls and broader guardrails, suggest LangChain sees the gateway as more than a stopgap. If the company delivers on those plans, the gateway could become a central piece of the agent development lifecycle, not just a runtime safety net.
For now, the public beta is open, and the company is inviting teams to try it. The path is clear: log into LangSmith, select "LLM Gateway" in the sidebar, and start configuring controls. The question is whether teams will adopt it quickly enough to avoid the next four-figure invoice.