Technology6 min read

High performance policy routed IPsec mesh SD-WAN

By · Published by Everything Blog

In short

ranet3 is a high-performance IPsec mesh SD-WAN solution that uses ESP in tunnel mode over UDP and IKEv2 with hybrid ML-KEM key exchange, featuring embedded Babel speaker and RTT metrics steering traffic through SRv6 by prefix and source/destination. It is interoperable with strongSwan, IKEv2, and ESP sessions with Ed25519 keys, and benchmarks show it outperforms wireguard-go, WireGuard, and IPsec kernel in terms of throughput, CPU usage, and latency. The solution supports userspace and kernel implementations, with userspace being slower but still faster than kernel implementations. It is designed to enhance performance and lower latency alongside existing mesh VPN and SD-WAN tools.

Key points

  • ranet3 is a high-performance IPsec mesh SD-WAN solution.: ranet3 is a high-performance IPsec mesh SD-WAN solution.
  • it uses ESP in tunnel mode over UDP and IKEv2 with hybrid ML-KEM key exchange.: it uses ESP in tunnel mode over UDP and IKEv2 with hybrid ML-KEM key exchange.
  • it features embedded Babel speaker and RTT metrics steering traffic through SRv6 by prefi…: it features embedded Babel speaker and RTT metrics steering traffic through SRv6 by prefix and source/destination.

Sources checked by our analysis pipeline: 1

..::----::.. .:-++++++++++++++-:. :=+++-:.. ..:-+++=: :=++-. .-+-. .=++-. :::::: . .::. :++=. :+#@@@@@@@@#+: *@@@@= :++= -#@@%*+-::-+*%@@#- :@@@@@@ .++= .*@@#- -#@@*. .#@@@@* -++. .#@@= =@@#. -++: .++- *@@- :++: -@@* :++. .@@# -%@@@@%- #@@. .--: -++ .++: %@@@@@@% +@@: ++- -++ %@@@@@@% +@@: ++- :++. .=+-. -%@@@@%- #@@. .++: .++- :%@@@%: :++: -@@* -++. -++. =@@@@@= =@@#. .++- .++= +%@%+ -#@@*. =++. :++= =*+-::-+*%@@#- =++: :++=. .#@@@@@@@@#+: .=++: .=++-. :::::: .-++=. :=++-. .-++=: :=+++-:.. ..:-+++=: .:-++++++++++++++-:. ..::----::..

ranet3

High performance policy routed IPsec mesh SD-WAN

- Transport

- Userspace ESP (RFC 4303) in tunnel mode over UDP (RFC 3948)

- Handshake

- IKEv2 (RFC 7296) with hybrid ML-KEM (FIPS 203) key exchange (RFC 9370)

- Routing

- Embedded Babel speaker (RFC 8966) with RTT metrics (RFC 9616) and source specific routes (RFC 9079)

- Steering

- SRv6 (RFC 8986) by source/destination prefix

- Source

- github:ranet3/ranet3

- License

- FSL-1.1-ALv2 / Apache-2.0

- mesh@ranet3.com

- Powering

- AS10779 / AS18932

Why ranet3

Standard protocols

Peers only need standard IKEv2, ESP, Babel and SRv6.

Interoperability is checked against:

- strongSwan

- IKEv2 and ESP sessions with Ed25519 keys

- BIRD

- Babel routes with RTT metrics, matched route for route

- Linux

- Kernel seg6local acts on the SRv6 headers ranet3 writes

Any other router or switch that supports IKEv2 with Ed25519 keys, Babel and SRv6 should interoperate as well.

Common mesh VPN and SD-WAN tools (Tailscale, NetBird, ZeroTier, Nebula, etc.) admit only nodes that run their own agent, and ranet3 runs beside them on the same host.

Latency routing

Babel measures the round trip of every link and routes through a relay whenever the direct link is slower.

RTT is measured continuously, and packet loss and outages count against a link. Traffic always takes the fastest path, moving back to the direct link once it is faster again.

Segment routing

A policy on source and destination prefix sends traffic through chosen waypoints and out at a chosen exit. Segment routing is implemented in userspace and works even on macOS, whose XNU kernel has no support for it.

Waypoints can come in any order the user specifies and may repeat, e.g. A->B->C->B->D with D as the exit.

Benchmarks

Dataplane | Runs in | Protocol and cipher |

---|---|---|

ranet3 | Userspace | ESP in UDP, AES-256-GCM |

wireguard-go | Userspace | WireGuard, ChaCha20-Poly1305 |

WireGuard | Kernel | WireGuard, ChaCha20-Poly1305 |

IPsec | Kernel (xfrm) | ESP in UDP, AES-256-GCM |

Medians of 3 runs under saturation

Head to head

Dataplane | Throughputhigher is better | CPU per Gbit/slower is better | Latency p99lower is better |

---|---|---|---|

wireguard-go user | 1.2-1.5x better | 0.8-0.9x better | 0.2-1.0x mixed |

WireGuard kernel | 1.6-2.1x better | 0.6x better | 1.9-2.4x worse |

IPsec kernel | 1.8-3.9x better | 1.1-1.7x worse | 1.6-2.0x worse |

Each range is ranet3's result divided by the other dataplane's across the 6 tests below.

Kernel and userspace

Userspace dataplanes copy every packet into their own process and are usually expected to be slower than kernel ones.

1 stream, Gbit/s | One way | Both ways | Share of lineone way |

---|---|---|---|

Line limit (10 GbE) | 8.84 | 18.76* | 100% |

Kernel, best of 2 | 5.35 | 7.46 | 61% |

Userspace, wireguard-go | 6.77 | 10.78 | 77% |

Userspace, ranet3 | 8.39 | 14.87 | 95% |

* Plain TCP without tunnel

Kernel WireGuard stays at 5.1 to 5.4 Gbit/s one way.

Kernel IPsec stays at 4.0 to 4.7 Gbit/s because its single flow lands on 1 NIC receive queue.

Full results

ranet3user | wireguard-gouser | WireGuardkernel | IPseckernel | |

---|---|---|---|---|

1 stream, out | 8.39 | 6.77 | 5.35 | 4.67 |

1 stream, in | 8.37 | 6.95 | 5.10 | 4.44 |

1 stream, both | 14.87 | 10.78 | 7.46 | 4.67 |

8 streams, out | 8.52 | 6.06 | 5.14 | 3.97 |

8 streams, in | 8.48 | 6.72 | 5.12 | 4.06 |

8 streams, both | 15.79 | 10.33 | 7.41 | 4.10 |

ranet3user | wireguard-gouser | WireGuardkernel | IPseckernel | |

---|---|---|---|---|

1 stream, out | 0.909 | 1.079 | 1.511 | 0.672 |

1 stream, in | 0.925 | 1.092 | 1.525 | 0.664 |

1 stream, both | 0.898 | 0.996 | 1.504 | 0.525 |

8 streams, out | 0.996 | 1.213 | 1.740 | 0.845 |

8 streams, in | 0.990 | 1.182 | 1.748 | 0.866 |

8 streams, both | 0.948 | 1.087 | 1.500 | 0.620 |

In the "both" rows kernel IPsec carried only 4.67 and 4.14 of the 6 Gbit/s offered.

ranet3user | wireguard-gouser | WireGuardkernel | IPseckernel | |

---|---|---|---|---|

1 stream, out | 2.01 | 2.10 | 1.07 | 1.04 |

1 stream, in | 1.91 | 2.18 | 0.85 | 1.23 |

1 stream, both | 3.17 | 3.15 | 1.65 | 1.76 |

8 streams, out | 2.77 | 16.40 | 1.26 | 1.53 |

8 streams, in | 2.86 | 5.09 | 1.17 | 1.50 |

8 streams, both | 4.96 | 7.69 | 2.48 | 2.51 |

Setup

Hardware

- Hosts

- 2 x Dell PowerEdge C6420

- Processor

- Intel Xeon Gold 6130 with 16 cores and 32 threads

- Memory

- 192 GiB

- NIC

- Intel X710

- Connector

- SFP+

- Bandwidth

- 10 Gbit/s each way

- Switch

- Both hosts on the same Dell S5296F-ON

Software

- OS

- Debian 13

- Kernel

- Linux 6.12.111

- wireguard-go

- 0.0.20250522

- wireguard-tools

- 1.0.20260223

- iproute2

- 7.1.0

- iperf3

- 3.21

Method

- Traffic

- TCP with cubic at inner MTU 1400

- Streams

- 1 and 8

- Load

- Saturated for throughput and latency and 3 Gbit/s each way for CPU cost

- Direction

- One way in each direction and both ways at once

- Runs

- Median of 3 runs of 10 seconds with the dataplanes interleaved

- CPU cost

- Busy cores from /proc/stat summed over both hosts with every dataplane unpinned

- Line limit

- 8.84 Gbit/s of TCP payload one way after ESP in UDP framing and 18.76 Gbit/s both ways without a tunnel

Original source: ranet3.com

Technology