Agent Substrate: Sleeping AI Agents on Kubernetes

Diagram of Agent Substrate suspending and resuming agents on Kubernetes

Agent Substrate is an open-source runtime that lets AI agents sleep on your own Kubernetes cluster. It writes an idle agent’s entire sandbox to object storage, then brings it back on whichever worker has room once another request comes in. A first hands-on run with a small LangGraph agent resumed a suspended session in 0.93 seconds, including the call to Claude.

Why agents need to sleep

Agents are idle far more than they’re busy: a burst of work lasting seconds, then a wait for the next message, timer or event. Sharing one container among many sessions, with state in a database, keeps that cheap. The bill climbs when every user or task needs a dedicated sandbox, say for executing code or holding files across runs, since nearly all of those machines sit unused.

Azure Container Apps Sandboxes use the same idea as a managed service: microVMs that suspend when idle and resume with their disk. Agent Substrate is maintained by Google and Solo.io, is at version 1.0, and runs on a cluster you control. Its docs put resume time under 500 ms.

Architecture of Agent Substrate with one request to a sleeping agent. On the left, your side: a client that sends HTTP with the header ate-target-actor set to lg-demo/conv-1, and the kubectl ate CLI. In the middle, Agent Substrate: step 1, atenet-router receives the request; step 2, ate-api-server picks a worker with room and resumes the actor; step 3, atelet restores the snapshot into a sandbox on a worker pod; a WorkerPool with two worker pods, one running conv-1, one with free room; step 4, atenet-egress, which all outgoing traffic passes. On the right, snapshot storage with suspended actors, and api.anthropic.com.
Image: baeke.info

gVisor or microVM

Ordinary containers use the node’s Linux kernel, so a flaw there can let code break out, which is a real concern when the code came from a language model. Each WorkerPool chooses between two barriers:

  • gVisor, Google’s open-source project, puts a small kernel between the agent and the host. It runs on any ordinary Linux node.
  • A microVM, built on Kata Containers and Cloud Hypervisor, gives each agent a small virtual machine and a kernel of its own. That isolates better, though nodes must expose hardware virtualization (/dev/kvm) and some cloud VM sizes don’t.
How the five concepts relate in the demo. Left, the Substrate control plane with records in PostgreSQL, managed with kubectl ate: the atespace lg-demo holds the ActorTemplate langgraph (image langgraph-agent, probe GET /readyz, cpu 1 and 512Mi memory, MEMORY snapshots, workerSelector workload=langgraph) and two actors created from it, blog-1 in state RUNNING and conv-1 in state SUSPENDED. Right, the Kubernetes cluster with the WorkerPool langgraph, a CRD with the label workload=langgraph and the gvisor class, and two worker pods: blog-1 runs in a gVisor sandbox in worker pod 1 while it is awake, worker pod 2 has free capacity. Bottom, object storage, an S3 or GCS bucket or RustFS on kind, with the golden snapshot of langgraph, saved once when the template was created, and the snapshot of conv-1, which holds memory and file system and is written on suspend and read on resume. A new actor starts from the golden snapshot, and suspended actors use no worker.
Image: baeke.info

Five concepts to learn

An actor is a single live instance of an agent, for example one conversation, and it is the unit that gets suspended and resumed. An ActorTemplate defines it: the image, the resources, the volumes and the depth of each snapshot. Registering a template launches the container a single time and stores a golden snapshot that new actors start from.

An atespace is the boundary templates and actors live in. A WorkerPool is a group of worker pods started in advance, and actors fill them until CPU and memory are used up. A snapshot is the saved state: volumes only, volumes plus file system, or those plus process memory.

Of the five, just the WorkerPool is a native Kubernetes resource. The rest are rows in Substrate’s PostgreSQL database, which is why plain kubectl get won’t list them and the kubectl ate plugin is required.

The flow of one model call with key injection. The actor, LangGraph in a gVisor sandbox: step 1, DNS resolves only hosts the egress policy lists; step 2, the TLS handshake, where the actor accepts the gateway's certificate because Substrate mounts the gateway CA in the sandbox; step 3, the SDK sends the request with the header Authorization: Bearer placeholder. The egress gateway terminates TLS for hosts in an https rule; step 4, it matches the rule for api.anthropic.com with a replaceHeaders effect; step 5, it fetches the credential from the credential provider, which checks that atespace lg-demo may read namespace lg-secrets. Step 6, the gateway sends the request to api.anthropic.com with the real key over a new TLS connection. You configure three things: the egress policy, a Kubernetes Secret with the key, and the credential provider's policy.
Image: baeke.info

Does the agent remember after a nap?

Any container that serves HTTP can be an actor, so nothing LangGraph-specific is required. The test agent was a one-node LangGraph graph behind FastAPI, with its history held only in process memory. The template set memory-level snapshots so that history would survive.

The test told the agent a name, suspended the actor and asked for the name again. The actor had no worker pod while suspended. The router woke it, the snapshot was restored, and the agent answered correctly. The round trip took 0.93 seconds on a laptop running kind.

The snapshots went to RustFS, an S3-compatible store the quickstart installs inside the cluster. The bundled plugin also works with Google Cloud Storage and anything speaking the S3 API. Azure Blob Storage isn’t covered, though a community plugin exists.

Keeping the API key away from the agent

A key inside the agent can be coaxed out by prompt injection, and memory snapshots would copy it into object storage as well. Substrate sidesteps both problems. The agent sends the string “placeholder” as its key, and the egress gateway swaps in the real one on the way out. To read the header, the gateway terminates TLS with a certificate it signs, so the template mounts the gateway’s CA and the Anthropic client trusts it.

The setup takes three pieces: an egress policy limiting the actor to api.anthropic.com, an ordinary Kubernetes Secret holding the key, and a credential provider policy. That provider denies everything by default, so the atespace must be explicitly allowed to read the secrets namespace.

Should you use it yet

The tester doesn’t run Agent Substrate in production and calls it early for most customers, many of whom would pick a managed Azure service anyway. For teams that must keep agents on their own hardware, the next step is trying the microVM option on nodes with KVM and watching how snapshot size affects resume time.

Source: baeke.info

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.