Artificial Intelligence1 min read

Agent Substrate: Advancing Kubernetes for Agents at Scale

By · Published by Everything Blog

In short

Traditional cluster configurations struggle with artificial intelligence agents because dedicating individual Pods wastes expensive memory during prolonged idle periods, while tearing them down leads to multi-second cold starts. To eliminate this trade-off, the runtime snapshots inactive sessions out of microVMs or gVisor sandboxes and restores them directly onto available workers without standard scheduling delays. The system targets three key areas, including coding assistants, reinforcement learning post-training workloads, and research tasks compiling CUDA code. It operates within the DSX AI Factory Platform alongside the KAI Scheduler and NVIDIA Dynamo.

Key points

  • The runtime bypasses standard Kubernetes scheduling to instantly restore agent sessions w…: The runtime bypasses standard Kubernetes scheduling to instantly restore agent sessions without multi-second cold starts.
  • NVIDIA is contributing upstream to prevent agentic infrastructure from splintering into c…: NVIDIA is contributing upstream to prevent agentic infrastructure from splintering into closed, proprietary runtimes.
  • Supported use cases span persistent coding assistants, bursty reinforcement learning envi…: Supported use cases span persistent coding assistants, bursty reinforcement learning environments, and shared GPU research sandboxes.

Sources checked by our analysis pipeline: 1

AI agents spend most of their time idle, waiting on a human prompt, a tool result or a model response. Kubernetes wasn’t built for that. Running one Pod per session holds memory for sessions that aren’t doing anything, and deleting Pods between turns pays a multi-second cold start every time and loses the agent’s state. RL post-training hits the same wall.

Agent Substrate, now donated to the Cloud Native Computing Foundation (CNCF) Sandbox, is an open-source, security-hardened, zero-trust runtime for sandboxed agents, engineered to safely run millions of agents and tools with 10x higher density than standard containers. It multiplexes many sessions over a small pool of worker Pods. Idle sessions are checkpointed, memory and filesystem included, and restored onto whichever worker is free when the next request arrives. As agentic infrastructure fragments into proprietary runtimes that bypass Kubernetes, NVIDIA is contributing upstream to keep this work open.

This post covers which workloads suit Agent Substrate, what is being built upstream, and how the project works with KAI Scheduler and NVIDIA Dynamo components in the DSX AI Factory Platform.

The…

Artificial Intelligence