How do you sandbox AI agents before they touch production?

Last Updated: 2026-11-06

Key takeaways

• ▪ Google donated Agent Substrate, gVisor and OpenRL to the CNCF in Sandbox status, announced on 5 November 2026 ahead of KubeCon North America.
• ▪ Agent Substrate separates an agent's state from the compute that runs it, so an idle agent can be snapshotted and resumed in under 500 milliseconds instead of holding a pod all day.
• ▪ A typical agent is idle 90% to 95% of the time, which is why dedicated pods become unaffordable at 10,000 active users and scale-to-zero cold starts break the experience.
• ▪ The documented limits matter more than the launch post. There is no GPU passthrough, gVisor cannot snapshot live CUDA contexts, and open database or MCP connections are dropped on suspend.
• ▪ CNCF Sandbox is the foundation's experimental entry tier. It signals neutral governance, not production readiness.
• ▪ A sandbox protects the host. It does not protect you from an agent doing something permitted but wrong. Identity, least privilege, egress control, human approval and audit are separate gates.

You sandbox an AI agent by putting it inside a boundary that cannot reach the host kernel, the corporate network or another tenant's data, then giving it only the tools and credentials that the task needs. Google's donation of Agent Substrate and gVisor to the CNCF makes that boundary easier to standardise, but the donation is not a certification. Agent Substrate is at Sandbox maturity, its documentation lists limits that change your design, and the hard part is not the sandbox at all. It is the governance around it: identity, permissions, egress, approvals and audit. This post walks through what Google shipped, where the boundary stops, and the five gates we would put in front of any agent before it touches production.

What is Agent Substrate?

Agent Substrate is an open-source Kubernetes runtime for AI agents that separates an agent's state from the compute that runs it. An Actor represents one agent session, with its own filesystem, environment variables, memory state and network rules. A Worker is a pre-warmed Kubernetes pod in a shared pool. When a request arrives, an L7 proxy called atenet-router parks it, the controller places the Actor on a free Worker, and the Actor's memory and disk state restore from tiered storage in under 500 milliseconds. When the turn ends, Substrate snapshots the state and frees the Worker for another agent. The runtime pairs with gVisor, Google's application kernel written in Go that intercepts system calls in user space, or with Cloud Hypervisor microVMs when a workload needs hardware virtualisation.

Google built it because Kubernetes assumes a program boots, binds a port and keeps serving. Agents do not work that way. They execute a step, call a model, and wait for tokens or a human. The donation, announced on 5 November 2026 ahead of KubeCon North America in Salt Lake City, moves Agent Substrate, gVisor and OpenRL into CNCF governance in Sandbox status. gVisor's own blog says its application was accepted on 28 September, after a September 7 submission and a September 22 review.

Why do agents need a different sandbox from containers?

Agents create two infrastructure problems that standard Kubernetes was not designed to solve. The first is idle waste. During a typical session an agent spends 90% to 95% of its time waiting for model inference, an external API or a human review. It consumes no CPU while it waits, but you cannot kill the container because it holds live state: language runtimes, local Git repositories, uncommitted edits and tool sessions. Keep a pod running all day and you pay for idle compute around the clock, which becomes unsustainable at 10,000 active users. Scale to zero and the cold start forces the container to clone repositories and reinstall packages, pushing wake-up latency past 20 seconds. The second problem is control plane overload. Reinforcement learning and evaluation suites run untrusted agent code across thousands of tasks at once, and creating a Kubernetes pod per sandbox overwhelms etcd, the API server and the IP address pool.

Agent Substrate's answer is to make the sandbox a first-class object rather than a pod. Actors hold state, Workers provide compute, and the controller multiplexes thousands of activations, suspensions and resumes per second without creating new pod objects. The design target is sub-second wake-up with full memory and disk state intact. That is a real architectural shift, and it is the reason the project matters beyond Google.

What does the CNCF donation actually change?

It changes ownership, not maturity. CNCF Sandbox is the foundation's entry tier for experimental projects. It gives the project vendor-neutral governance and a path to broader contribution, but it does not mean the runtime is production ready, audited for your threat model, or supported by every cloud. For a platform team, the practical effect is that the boundary is no longer tied to one vendor's roadmap, and the API can be adopted by other cloud providers, hardware vendors and independent software companies. The technical limits are still yours to verify. Google's own documentation is the place to start, because it is more specific than the launch post.

Question	Agent Substrate (open source)	Agent Compute Engine (managed)
Isolation	gVisor software kernel or Cloud Hypervisor microVM	Dedicated microVM with Titanium attestation
Credential custody	You design it, the sandbox does not hold secrets by default	Credentials kept outside the sandbox
GPU inside the sandbox	No documented passthrough	Not stated in launch material
Maturity	CNCF Sandbox, 1.0 release, experimental tier	Preview promised in the coming months, no date or price
Where it runs	GKE Standard 1.37 or later, self-managed clusters	Google Cloud managed service

Where does the sandbox stop?

The boundary stops at the documented limits, and those limits decide whether the architecture fits your workload. Agent Substrate does not support GPU device passthrough into actor containers, and gVisor cannot snapshot live CUDA contexts, so model inference has to run outside the sandbox. Open network connections, including database sessions and connections to MCP servers, are not preserved when an agent suspends, so agent code must reconnect after it resumes. Supported clusters are GKE Standard running version 1.37 or later. Module documentation for version 0.5.0 advises a larger kernel neighbour table on Linux hosts that run more than about a thousand sandboxes. And gVisor is not immune to attack: a CVE tracked as CVE-2026-96812 scored 8.8 in CVSS v4.0, which is a reminder that a software kernel reduces exposure rather than eliminating it. Teams that need a hardware boundary can choose Cloud Hypervisor microVMs through the same API.

There is also a documentation inconsistency worth knowing before you design. Google's 5 November blog describes gVisor as supporting GPUs and checkpoint/restore natively, while the Substrate documentation says live CUDA contexts cannot be snapshotted. Check which statement applies to your workload rather than assuming the launch post is the contract. If your agent needs a GPU inside the sandbox today, there is no supported path, so inference belongs outside the boundary.

What are the five gates before production?

A sandbox answers one question: can this code damage the host or reach another tenant? It does not answer whether the agent should be allowed to send the email, delete the record or approve the invoice. We put five gates in front of any agent that touches a live business workflow. First, identity: every agent gets its own workload identity and short-lived credentials, never a shared human login. Second, least privilege: tool allowlists, read-only defaults and scoped tokens, with no wildcard access to a file store or an inbox. Third, egress: default deny, an allowlist of domains and services, and a log of every outbound call, including calls to MCP servers. Fourth, human approval: irreversible actions such as payments, deletions and external communications require a preview and an explicit approval. Fifth, audit and replay: every prompt, tool call, file write and approval is logged against an actor ID, retained to your records obligations, and replayable when something goes wrong.

The gates are not optional extras. Prompt injection can make an agent do something that is technically permitted and still wrong. The sandbox will not stop it, because the action is inside the boundary. Permissions and approvals stop it. That is the difference between isolation and governance, and it is where an enterprise architect earns their keep.

What should a platform team do this quarter?

Start with a threat model, not a vendor shortlist. Write down what the agent can reach, what it can change, and what an attacker could do with its credentials. Then choose the boundary: gVisor for development and internal tools where a software kernel is proportionate, Cloud Hypervisor microVMs for regulated production workloads that need a hardware boundary. Pilot one workflow with a clear blast radius, such as a read-only support triage agent, and instrument four numbers: resume latency, blocked egress attempts, approval rejection rate and the share of tool calls that needed escalation. Red-team the pilot with a prompt injection that tries to exfiltrate a file or call an unapproved endpoint. If the sandbox holds and the gates fire, you have evidence for the next workflow. If they do not, you have found the gap before a customer did.

For the testing side of the same problem, see our earlier post on how to test an agent before it touches a live workflow. For the accountability question that follows, see who is accountable when an agent makes a costly mistake.

Frequently asked questions

What is Agent Substrate?

Agent Substrate is an open-source Kubernetes runtime for AI agents that separates an agent's state from the compute that runs it. An Actor holds the session's filesystem, environment and memory, while a Worker is a pre-warmed pod from a shared pool. The controller restores an Actor onto a Worker in under 500 milliseconds and snapshots it back to tiered storage when the turn ends. It pairs with gVisor or Cloud Hypervisor microVMs for isolation and was donated to the CNCF in Sandbox status in November 2026.

Is Agent Substrate production ready?

Not by default. CNCF Sandbox is the foundation's experimental entry tier, so the donation signals neutral governance rather than production readiness. The runtime has a 1.0 release and documented limits, including no GPU passthrough, no live CUDA snapshots, dropped network connections on suspend and a GKE 1.37 or later requirement. Treat it as a platform component to evaluate against your own threat model, not as a turnkey service.

Do I need Kubernetes to sandbox an AI agent?

No. Kubernetes helps at scale, but a single agent can be sandboxed with a container runtime, a microVM or a separate host with no inbound network path. The principles are the same: keep the agent away from the host kernel, give it only the tools and credentials it needs, deny egress by default and log every action. Agent Substrate matters when you need thousands of stateful agents with sub-second resume times.

What is the difference between Agent Substrate and Agent Compute Engine?

Agent Substrate is the open-source Kubernetes runtime that runs agents in gVisor or Cloud Hypervisor sandboxes and manages their state. Agent Compute Engine is Google's managed service that gives each agent its own microVM with credentials kept outside the sandbox and attestation keys held in Titanium hardware. The managed service is promised for preview in the coming months with no date or price. The open runtime is available now and puts the operational burden on your team.

Can an agent sandbox use a GPU?

Not inside an Agent Substrate actor today. Google's documentation lists no GPU device passthrough, and gVisor cannot snapshot live CUDA contexts, so inference must run outside the sandbox. Design the architecture so the agent calls a model endpoint rather than holding the GPU itself. If a workload genuinely needs a GPU inside the boundary, a Cloud Hypervisor microVM is the supported option to investigate.

What is the biggest risk in sandboxing an AI agent?

Treating the sandbox as the whole control. Isolation protects the host and other tenants, but it does not stop an agent from doing something permitted and wrong, such as emailing the wrong customer or deleting a record it was allowed to touch. Identity, least privilege, egress control, human approval and audit are separate gates. The biggest risk is shipping an agent with a strong boundary and weak permissions.