Technology1 min read

Live-migration support in FreeBSD and bhyve coming

By · Published by Everything Blog

In short

The technology uses an iterative pre-copy mechanism over TCP that repeatedly transmits dirty-page memory before a brief final pause sends device and CPU states. Built on experimental snapshot capabilities, the system incorporates dirty-page tracking directly into the vmm kernel module and coordinates transfers using bhyve and bhyvectl. The source host maintains safety by resuming the original guest if an error occurs before the destination validates and commits the transfer. Currently presented as a work-in-progress request for comments, the feature cannot yet migrate guests configured with PCI passthrough.

Key points

  • System safety is maintained by allowing the source host to safely resume the original gue…: System safety is maintained by allowing the source host to safely resume the original guest if a network or destination failure happens before the final commit.
  • Virtual machine configurations using PCI passthrough are currently unsupported because th…: Virtual machine configurations using PCI passthrough are currently unsupported because the migration process depends entirely on snapshot-capable emulated devices.
  • The new capability provides essential infrastructure for host maintenance and supports pl…: The new capability provides essential infrastructure for host maintenance and supports platforms such as BoxyBSD and the Proxmox-like Sylve project.

Sources checked by our analysis pipeline: 1

This project presents a first working state for live migration of bhyve guests on FreeBSD 15.1. A running amd64 guest can move between two hosts over TCP while continuing to run; it is paused only for the final stop-and-copy phase. The work builds on bhyve's experimental snapshot support and adds dirty-page tracking in the vmm kernel module together with a migration protocol in bhyve and bhyvectl.

I previously discussed the state of bhyve live migration, its historical development, existing limitations, and why it matters for the future of FreeBSD virtualization in my article bhyve on FreeBSD and VM Live Migration – Quo vadis?.

The goal is deliberately practical: make it possible to test, discuss, and improve a real implementation rather than discuss live migration only in the abstract. The current code and test tooling are being published as a WIP RFC so that the next steps can be shaped in the open. A more detailed demonstration of a VM being live migrated can be watched on my YouTube channel.

Why this matters

Live migration is an important building block for…

Technology