NVIDIA Snapshot, a new open-source component of the NVIDIA DSX AI Factory Platform, makes initialized GPU workloads reusable across inference replicas—speeding startup and recovery.
When inference demand spikes or a replica fails, available GPUs don’t translate immediately into serving capacity. Initializing an LLM workload—a process known as a cold start—can take minutes, delaying scaling and recovery.
Snapshot is a checkpoint and restore system for GPU workloads on Kubernetes. It captures a fully initialized workload—including CPU and GPU memory—and restores that state on compatible infrastructure, bringing inference capacity online in seconds instead of minutes. Figure 1 shows the startup sequence. Snapshot skips repeated workload initialization; infrastructure provisioning and container image loading remain outside the measured restore path.
This post explains how Snapshot checkpoints and restores GPU workloads, how platform teams can integrate it, and how restore times compare with cold starts across three inference engines.
Snapshot for AI factory operations
Originally developed for NVIDIA Dynamo inference workers, Snapshot provides checkpoint and restore capabilities that other inference engines, schedulers, controllers, and platforms can integrate.
Snapshot provides two operations for GPU Pods:
-…