Kong AI Gateway 2.0 on Google Cloud: Securing GKE, Cloud Run, and Vertex AI(Agent Platform)

Most teams running on Google Cloud don't pick one compute model and stay there. Some services live...

Most teams running on Google Cloud don't pick one compute model and stay there. Some services live on GKE because they need fine-grained control, sidecars, or GPU scheduling. Others run on Cloud Run because a stateless HTTP API doesn't need a cluster to babysit it. Increasingly, a third category has joined the mix: LLM calls, MCP tool servers, and autonomous agents, each with its own traffic patterns and risk profile.

Enterprises are rapidly shifting from conventional request-response APIs to composite, AI-native ecosystems. Standard REST and GraphQL endpoints are no longer just consumed by frontends; they are dynamically queried as tools by autonomous agents, LLM pipelines, and Model Context Protocol (MCP) servers.

At the same time, compute is split across Google Cloud platforms. Microservices run on Google Kubernetes Engine (GKE), while serverless APIs run on Cloud Run.

his fragmentation introduces a critical infrastructure question: How do you enforce uniform authentication, observability, and cost governance across GKE and Cloud Run without maintaining duplicate security stacks?

Kong AI Gateway 2.0 addresses this by treating AI primitives—models, providers, agents, and MCP servers—as first-class control plane entities rather than generic HTTP plugins.

Why AI Traffic Breaks Traditional API Gateways

Standard API gateways evaluate traffic through a deterministic lens: Client → Request → Upstream API → Response.

AI architectures, by contrast, execute non-linear, agentic loops:

Managing this traffic requires capabilities that standard proxy layers lack:

Token-based rate limiting: Quotas enforced by input/output token volume rather than raw request counts.

Semantic protection: Real-time prompt inspection, model routing, and credential abstraction.

Dynamic tool governance: Securing runtime discovery of tools, not just their direct HTTP invocations.

The AI Gateway 2.0 Abstraction Model

Kong AI Gateway 2.0 replaces downstream plugin attachment with dedicated architectural abstractions:

AI Model Providers & Models: Decoupled backends that manage upstream connection pools, fallbacks, and credential routing.

AI MCP Servers: First-class entities representing tool providers.

AI Agents & A2A Routing: Secure conduits for multi-agent negotiation.

AI Consumers & Policies: Identity-aware boundaries governing spend, prompt guardrails, and access rights.

By decoupling configuration (Kong Konnect) from the data plane, runtime operations scale independently across private networks without exposing configuration planes to direct data streams.

Deploying Across Google Cloud Runtimes

Google Kubernetes Engine (GKE)

GKE serves as the backbone for complex, stateful microservices, internal agents, and orchestration engines.

Kubernetes-Native Config: Deploy the data plane via the Kong Kubernetes Operator (KKO) and configure models or policies directly via CRDs.

Workload Identity Federation: The data plane inherits Google Cloud IAM roles via Workload Identity, eliminating static service account keys when authenticating upstream.

Cloud Run

Cloud Run provides high-density execution for event-driven webhooks, ephemeral MCP tools, and lightweight inference endpoints.

Centralized Security Layer: Keep Cloud Run services locked to private internal ingress, routing all inbound client and agent requests through Kong AI Gateway for authentication and logging.

Unified Surface: Both GKE-based services and Cloud Run endpoints exist under the same consumer-facing namespace.

Integrating Vertex AI and Gemini Models

Instead of distributing Google Cloud service account keys to individual applications, route model consumption through Kong AI Gateway:

IAM-Backed Egress: The gateway manages OAuth token minting via GCP Workload Identity to interact directly with Vertex AI standard and enterprise endpoints.

Virtual Models & Fallbacks: Configure fallback chains (e.g., primary Gemini Ultra fallback to Gemini Flash) at the gateway level without changing client-side code.

Identity-Aware Routing: Route internal teams to standard tiers while allocating high-throughput, low-latency allocations to mission-critical applications.

Advanced AI Controls: MCP Bundling & Agent-to-Agent Security

MCP Server Bundling & Scoped Discovery

Rather than having an agent establish connections to dozens of independent tool endpoints, Kong aggregates tool namespaces into a unified endpoint:

Discovery Filtering: The gateway intercepts tool list negotiations. If an agent lacks permissions for the billing tool, that capability is stripped from the MCP discovery response, preventing hallucinated or unauthorized executions.

Agent-to-Agent (A2A) Governance

When specialized agents communicate across microservice boundaries, Kong AI Gateway establishes mutual authentication, traces execution graphs, and applies rate-limiting policies to prevent unconstrained recursion loops:

Production Multi-Runtime Architecture

Deploying Kong AI Gateway 2.0 across Google Cloud isolates runtime infrastructure choices from AI governance requirements. GKE handles container orchestration, Cloud Run provides elastic serverless execution, and Vertex AI powers model inference. Kong AI Gateway 2.0 ties the ecosystem together providing a single control plane to secure and route modern AI traffic.