AI Agents and Agent Gateways on Kubernetes¶
An agent gateway is a proxy between AI agents and everything they call: language models, MCP tool servers, and other agents. It authenticates each agent, decides which tools it may use, enforces token budgets, and records every call. This page covers running agents on Kubernetes with an agent gateway in front and sandboxes for the code they write.
What You'll Learn¶
- What changes when the workload is an agent rather than a request/response service
- What an agent gateway does for LLM, MCP, and agent-to-agent (A2A) traffic
- How to put an MCP server behind agentgateway on Kubernetes
- How to run code that agents write with Agent Sandbox and Agent Substrate
Why This Matters¶
An agent is a loop: it calls a model, decides which tools to use, and acts through MCP servers, internal APIs, and other agents. Every hop carries credentials, and every piece of text the agent reads can try to steer it (see prompt injection). Without one place to apply policy, each agent team builds its own authentication, rate limits, and audit logging. MCP servers multiply, each holding its own API keys. Token spending has no owner until the invoice arrives.
Mental Model¶
Treat agents as untrusted, bursty clients with powerful tools. Give each one its own identity, send everything it calls through one policy point (the agent gateway), and run any code it writes in a sandbox that holds nothing worth stealing.
flowchart LR
A["Agent pod<br>own ServiceAccount, no provider keys"] --> AG["Agent gateway<br>authn, tool policy, token budgets, traces"]
AG --> L["LLMs<br>hosted APIs or an InferencePool"]
AG --> M["MCP servers<br>behind one virtual endpoint"]
AG --> O["Other agents<br>A2A"]
A --> S["Sandbox<br>gVisor or Kata, no credentials"]
How It Works¶
Agents are workloads with extra needs¶
An agent runs as an ordinary Deployment, but lock it down more than a typical service:
- Its own ServiceAccount, with
automountServiceAccountToken: falseunless it really needs the Kubernetes API. An agent that "helps debug the cluster" with cluster-admin rights is one injected instruction away from deleting it. Grant only what it needs with RBAC. - No provider API keys in the pod. The gateway holds them and injects them into outbound requests, so a compromised agent has nothing to leak.
- Default-deny egress. An agent that can fetch any URL can be talked into sending data anywhere. Allow only the gateway and DNS with a NetworkPolicy:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: support-agent-egress
namespace: agents
spec:
podSelector:
matchLabels:
app: support-agent
policyTypes: ["Egress"]
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: agentgateway-system
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
If you'd rather declare agents than write Deployments, kagent is a Kubernetes-native agent framework and a CNCF Sandbox project. Version 1.0 (November 2026) defines agents with CRDs in api.kagent.dev/v1alpha3, such as Agent, AgentTemplate, ModelConfig, and RemoteMCPServer, and runs them on Agent Substrate (described below).
The traffic an agent gateway sees¶
| Traffic | What it is | What the gateway can do with it |
|---|---|---|
| LLM calls | OpenAI- or Anthropic-style APIs to hosted providers or self-hosted models | Route between providers and models, fail over, count tokens, enforce budgets |
| MCP | The Model Context Protocol, how agents discover and call tools (build an MCP server) | Combine many servers into one endpoint, authorize per tool, audit every call |
| A2A | Agent-to-agent protocol for delegating tasks between agents (A2A lab) | Authenticate both sides, route by agent, trace across agents |
MCP's current specification (revision 2026-07-28) made the protocol stateless. It removed the initialize handshake and the Mcp-Session-Id session header, and every request over Streamable HTTP now carries MCP-Protocol-Version, Mcp-Method, and Mcp-Name headers. Load balancers and gateways can route and authorize on the tool name without parsing the JSON body, and any replica can serve any request. Servers still on a 2025 revision keep sessions inside one replica. Run those as a single replica, use session affinity, or upgrade them.
Agent gateways in November 2026¶
| Gateway | Status | Notes |
|---|---|---|
| agentgateway | Linux Foundation project; v1.7.0 (October 2026); Apache-2.0 | Proxy built for LLM, MCP, and A2A traffic. Implements the Gateway API (gatewayClassName: agentgateway) with AgentgatewayBackend and AgentgatewayPolicy resources. Also a conformant InferencePool implementation |
| Agent Router (formerly Envoy AI Gateway) | Agentic AI Foundation project; v1.3.0 (November 2026); Apache-2.0 | Built on Envoy Gateway. AIGatewayRoute for LLM providers, MCPRoute for MCP, QuotaPolicy for token budgets. Supports InferencePool. The CRDs keep the aigateway.envoyproxy.io group |
| GKE Agent Gateway | Preview (November 2026), GKE 1.37.2+ in the Rapid channel | Managed control of traffic to, from, and between agents on GKE |
Example: an MCP server behind agentgateway¶
Install the controller (it needs the Gateway API CRDs first):
kubectl apply --server-side -f \
https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.6.3/standard-install.yaml
helm install agentgateway-crds oci://cr.agentgateway.dev/charts/agentgateway-crds \
--version v1.7.0 --namespace agentgateway-system --create-namespace
helm install agentgateway oci://cr.agentgateway.dev/charts/agentgateway \
--version v1.7.0 --namespace agentgateway-system
Create the gateway that agents will call:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: agentgateway-proxy
namespace: agentgateway-system
spec:
gatewayClassName: agentgateway
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: All # tighten with a namespace Selector in production
Run the MCP server as a normal Deployment and Service. appProtocol: agentgateway.dev/mcp tells agentgateway that this port speaks MCP:
apiVersion: apps/v1
kind: Deployment
metadata:
name: tickets-mcp
namespace: mcp-tools
spec:
replicas: 2
selector:
matchLabels:
app: tickets-mcp
template:
metadata:
labels:
app: tickets-mcp
spec:
automountServiceAccountToken: false
containers:
- name: server
image: registry.example.com/tools/tickets-mcp:1.4.0 # your MCP server
ports:
- containerPort: 8000
---
apiVersion: v1
kind: Service
metadata:
name: tickets-mcp
namespace: mcp-tools
spec:
selector:
app: tickets-mcp
ports:
- port: 80
targetPort: 8000
appProtocol: agentgateway.dev/mcp
Then describe the MCP backend and route /mcp to it:
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
name: tickets
namespace: mcp-tools
spec:
mcp:
targets:
- name: tickets
static:
backendRef:
name: tickets-mcp
port: 80
protocol: StreamableHTTP
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: mcp
namespace: mcp-tools
spec:
parentRefs:
- name: agentgateway-proxy
namespace: agentgateway-system
rules:
- matches:
- path:
type: PathPrefix
value: /mcp
backendRefs:
- group: agentgateway.dev
kind: AgentgatewayBackend
name: tickets
Agents now use one MCP endpoint, http://agentgateway-proxy.agentgateway-system/mcp. Add more targets to the backend and agentgateway merges their tools into one virtual server. From here, add policy with AgentgatewayPolicy: OAuth or JWT authentication for MCP clients, CEL rules that allow a tool only for callers with certain JWT claims, and token budgets for LLM routes.
Budgets and per-tool authorization¶
Whichever gateway you choose, these are the policies that matter most:
- Authenticate every caller. MCP over HTTP uses OAuth-based authorization, and gateways also accept JWTs and API keys. Map each agent to its own identity, not a shared key.
- Authorize per tool, not per server. "The support agent may call
get_ticketandadd_comment, notclose_ticket." agentgateway expresses this as CEL rules over JWT claims, and Agent Router'sMCPRoutehas an authorization section that also uses CEL. - Budget tokens per agent and per tenant. agentgateway applies token-unit rate limits through
AgentgatewayPolicy, and Agent Router'sQuotaPolicycharges a CEL-defined cost per request against a budget, answering429once it's spent. A single agent stuck in a tool loop can otherwise burn a month of budget in a night. - Trace everything. Both gateways export OpenTelemetry traces and token metrics. That is where you see which agent called which tool with how many tokens. See AI Observability.
Running the code agents write¶
Coding and data-analysis agents run code they generated. That code must never run in the agent's own container, which has its identity and network access. Two Kubernetes projects target this:
Agent Sandbox is a Kubernetes SIG Apps project (v1.0.12, November 2026) for isolated, stateful, single-pod workloads. A Sandbox (agents.x-k8s.io/v1beta1) is one pod with a stable identity that can hibernate and resume. SandboxTemplate, SandboxClaim, and SandboxWarmPool (extensions.agents.x-k8s.io/v1beta1) let you stamp sandboxes out from a template and keep pre-warmed ones ready. Isolation comes from the pod's runtimeClassName:
apiVersion: agents.x-k8s.io/v1beta1
kind: Sandbox
metadata:
name: code-runner-7f3a
namespace: agent-sandboxes
spec:
podTemplate:
spec:
runtimeClassName: gvisor # or kata-qemu; the RuntimeClass must exist
automountServiceAccountToken: false
containers:
- name: runner
image: python:3.13-slim
command: ["sleep", "infinity"]
resources:
limits:
cpu: "1"
memory: 1Gi
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: deny-all
namespace: agent-sandboxes
spec:
podSelector: {}
policyTypes: ["Ingress", "Egress"]
Agent Substrate (Apache-2.0, v1.0.1, November 2026) is a runtime for very large numbers of mostly idle agents. Every agent runs in a gVisor or Cloud Hypervisor sandbox. Idle agents are snapshotted out of memory and resumed onto a small pool of warm worker pods when a request arrives. Each agent gets a short-lived SPIFFE identity, and an egress gateway injects credentials outside the sandbox, so the agent never holds them. kagent 1.0 uses it as its runtime.
| Agent Sandbox | Agent Substrate | |
|---|---|---|
| Unit | One isolated pod per sandbox | Many agents multiplexed onto fewer worker pods |
| Best for | Per-session code execution, development environments | Platforms hosting thousands of mostly idle agents |
| Isolation | Your choice of RuntimeClass (gVisor, Kata) | Always gVisor or a microVM |
On GKE, both became generally available in November 2026.
What Most Engineers Miss¶
- Prompt injection turns tool access into attacker access. Least privilege has to apply per tool, enforced outside the model. A system prompt saying "never call
close_ticket" is not a control. - Budgets are a reliability feature. Runaway agent loops are the LLM equivalent of a fork bomb. Per-agent token limits stop the damage. Alerts tell you after it's done.
- Agent egress is an SSRF problem. "Fetch this URL" tools can reach cloud metadata endpoints and internal services unless egress is default-deny.
- Gateways see every prompt. Their logs and traces contain whatever users and tools sent. Decide what gets stored before you turn on content capture.
- Older MCP servers are stateful. Before the 2026-07-28 revision, Streamable HTTP sessions lived inside one replica. Scaling such a server behind a plain Service produces random "session not found" errors.
Common Mistakes¶
- Giving an agent's ServiceAccount broad cluster permissions "so it can investigate problems".
- Mounting provider API keys into agent pods instead of letting the gateway inject them.
- Running agent-generated code inside the agent container.
- Sharing one powerful token between every agent and every MCP server.
- Shipping agents with no per-agent token budget.
Check Yourself¶
Hidden instructions
An agent with a file-reading tool reads a web page that contains hidden text telling it to upload ~/.kube/config to an external site. Which defense actually stops the upload?
- A larger model that is better at spotting manipulation
- No credentials in the agent pod, per-tool authorization, and default-deny egress
- A longer system prompt that forbids uploading files
Model behavior can be steered by text it reads. Controls enforced outside the model hold regardless of what the model decides.
Session not found
A team scales its MCP server from 1 to 3 replicas behind a Service. Clients now fail at random with "session not found". The server implements the 2025-06-18 revision of MCP and assigns session IDs. Why?
- The Service needs
appProtocol: http2 - Sessions in that revision live in one replica, so requests in a session must keep hitting it
- Three replicas exceed the gateway's connection limit
Use session affinity, run one replica, or move to the stateless 2026-07-28 revision, where any replica can serve any request.
Interview Questions¶
- What does an agent gateway do that a regular API gateway doesn't?
- How would you isolate code that an agent writes and executes on Kubernetes?
- How do you stop one runaway agent from spending the whole month's token budget?
- Design authentication and authorization for 20 agents calling 15 MCP servers.
Further Reading¶
- agentgateway Kubernetes docs
- Agent Router documentation
- Model Context Protocol specification
- Agent Sandbox and Agent Substrate
- kagent
Next¶
Continue to AI Observability to monitor the models and agents you've deployed, or move on to OpenShift.