How to run hundreds of AI agents on a couple of servers without going broke

Usually working with Kubernetes is predictable: one service — one pod (well, or a ReplicaSet). But when we move to the world of AI agents, the familiar pattern breaks down. Imagine you have thousands of agents, each holding a dialogue context in memory, environment variables, and file system state. Most of the time they do nothing — just wait for a user command or an event. Allocating a full pod in K8s for each one is expensive: the control plane will simply choke on that many objects, and cluster resources will be wasted.

The Google team (though the project is marked as unofficial) introduced Agent Substrate. This is a system that allows "juggling" hundreds of active agent sessions using just a few physical pods.

The problem with the standard approach

In Kubernetes, creating a pod isn't an instant operation. You need to update the state in etcd, wait for the scheduler, pull the image, and start the container. For interactive agents, a delay of a couple of seconds is already too much. Plus, when an agent "sleeps," you want to free up its resources, but when it wakes up, it needs to instantly restore all files and variables in memory.

Agent Substrate solves this through the concept of actors and workers. Actors are your applications (agents), and workers are long-lived pod-shells. The system maps a huge number of actors onto a small pool of workers.

Key features of the project

Instant session teleportation

This sounds like magic, but in practice it works based on gVisor. When an agent is inactive, Substrate takes a snapshot of its state (including memory) and saves it. As soon as a request comes in, the system finds a free worker and "unpacks" the snapshot there. Activation takes less than a second.

State preservation without hacks

You don't need to painfully configure external databases for every little thing an agent does or mount slow network drives. The entire state, including RAM contents and local files, is preserved perfectly. If an agent was writing code in the terminal or held a data array in memory, after hibernation it will continue exactly from where it left off.

Insane multiplexing

In the demo video, the authors show how 250 stateful actors fit on just 8 pods. This provides resource savings of tens of times. For those building a platform for AI agencies or development services (like Claude Code), this is a direct path to reducing cloud bills.

How it works under the hood

The project is written in Go and tightly integrated with the Kubernetes ecosystem, but it moves actor management out of the critical path of the K8s control plane.

Here are the main components:

• ateapi: The core of the system, manages the actor lifecycle.
• atelet: A daemon on nodes that orchestrates snapshots and state transfer.
• atenet: Handles traffic routing so requests reach the right worker where the needed actor is currently "awake."
• gVisor (runsc): Provides isolation and enables process checkpoints.

By the way, the project is completely framework-agnostic. It doesn't matter if you're using LangChain, Claude Code, or a custom Python script — if it's packaged as an OCI container, Substrate will run it.

Quick start

If you have Go, Docker, and kubectl installed, you can try the system locally via kind. The developers have prepared scripts that will spin up a cluster and demo environment in a couple of minutes.

# Создаем локальный кластер
hack/create-kind-cluster.sh

# Ставим систему и демо-приложение
hack/install-ate-kind.sh --deploy-ate-system
hack/install-ate-kind.sh --deploy-demo-counter

# Устанавливаем плагин для kubectl
go install ./cmd/kubectl-ate

# Создаем пространство и первого актора
kubectl ate create atespace demo
kubectl ate create actor my-counter-1 -a demo --template ate-demo-counter/counter

After that, you can call an actor with a standard curl. The system will automatically determine if it needs to wake it up and where to route the request.

Is it worth trying now

The project is in the "Very Early Development" stage. This means the API will definitely change, and backward compatibility is not guaranteed. It's too early for production, but for R&D departments and those designing infrastructure for LLM services, this is a must-have for study.

If you need to run thousands of isolated sandboxes for code execution or a horde of AI assistants that must remember context, Agent Substrate offers a much more elegant path than trying to scale native Kubernetes to infinity.

The video shows the system juggling actors in real time.

The project team actively communicates in CNCF Slack (channels #substrate-users and #substrate-dev) and holds weekly meetings. It's clear that serious expertise in virtualization and containerization stands behind this project. At minimum, it's a great example of how to extend K8s capabilities for specific workloads.