700 Clients, 400,400 Routes: rustbgpd vs BIRD vs OpenBGPD

rustbgpd delivers a full new-policy table to 700 route-server members in 1.28–1.57 s at p50 (worst observer 2.00 s). BIRD 3.3.1 takes 64–85 s. OpenBGPD 9.1 takes 244–251 s. That's roughly 50× and 170×.

Same harness, same host: 700 clients, 400,400 IPv4 routes, live churn throughout. Every cell is from the v0.64.0 same-host refresh on 2026-08-08 — two campaign runs, zero decode errors. rustbgpd has since tagged v0.65.0; the matrix hasn't been rerun at that tag yet.

The origin story — wire codec, FSM, and the first performance pass — is in Building a BGP Daemon in Rust.

The Harness

700 real BGP stub sessions over loopback TCP — real OPEN/KEEPALIVE/UPDATE wire exchange, per-member 572-prefix slices of a 400,400 × /24 base table. Eight members flap churn blocks every 125 ms throughout, about 64 UPDATE events per second, so nothing here is measured on an idle daemon.

Every stub decodes every UPDATE it receives and timestamps arrival. Completion requires each observer to hold every expected unique base prefix carrying that reload's policy-generation community, tracked as a per-observer bitmap. A duplicate never advances the window. Daemon-side clocks — rustbgpd's reload log line, BIRD's Reconfigured, bgpctl's request processed — are advisory and never appear in the tables.

Each daemon runs at its documented strongest configuration and reloads by its own operator-documented mechanism: SIGHUP for rustbgpd, birdc configure for BIRD, bgpctl reload for OpenBGPD. The policies are semantically identical but natively expressed — .rpol chains, BIRD filter functions, OpenBGPD deny/match rules — and the delivered generation is verified by sampling communities off decoded UPDATEs.

The Numbers

Reload, range over 8 reloads across both runs:

Metric	rustbgpd	BIRD 3.3.1	OpenBGPD 9.1
Stall p50	0.42–0.60 s	1.70–2.70 s	0.25–0.29 s
Completion p50	1.28–1.57 s	64.3–84.5 s	244–251 s
Completion worst	2.00 s	93.8 s	251.3 s

Cold start:

Metric	rustbgpd	BIRD 3.3.1	OpenBGPD 9.1
700 sessions Established	0.7 s	18.2–20.5 s	68.9–85.8 s
Full table to all observers	4.8–5.0 s	60.9–63.3 s	338–352 s

Flapstorm — 50 members hard-close, 650 survivors observe, then the 50 return:

Metric	rustbgpd	BIRD 3.3.1	OpenBGPD 9.1
Withdraw p50	0.31–0.47 s	0.47–0.61 s	10.45–11.47 s
Re-announce p50	0.49–0.55 s	2.85–3.74 s	21.14–21.54 s

rustbgpd is the only daemon in the matrix holding both a sub-second median stall and single-digit-second completion. Every one of the 700 members verifiably holds the full new policy within 2 s of the reload, worst case, in both runs.

Why the Reload Is Fast

The reload numbers come out of update groups, and specifically where the sharing happens.

BIRD and FRR share encoded packets between members of an update group. That works, but it forces the group key to include every negotiated encode option, which fragments groups. rustbgpd's layering is different: all per-peer wire preparation — NEXT_HOP rewrite, ORIGINATOR_ID/CLUSTER_LIST, GShut, LLGR, encode options — lives in the transport session, not in RIB staging. So rustbgpd shares staged routes one layer up, with a group key built only from RIB-staging inputs.

That matters because the export tail is the expensive part. Split-horizon, RFC 4456 suppression, family and LLGR gates, policy evaluation, equality diff, Adj-RIB-Out insert — before update groups, all of it ran once per (peer × prefix), and staging plus Adj-RIB-Out was 82.8% of route-reflector CPU at 256 clients. Sharing at the staged-route layer collapses that to once per group.

Then a second sharing pass on top. Members of a group get an Arc-shared announce inventory, but each session task was still encoding its own full wire stream from it — at 700 members that's byte-identical work repeated 700 times, and every observer's first post-reload UPDATE arrives only after the total encode time divided by core count. It showed up as a flat ~1.03 s stall, p50 ≈ p95, about 0.76 s of it encoding alone. Update-group membership already guarantees a shared export-policy outcome, and the session export profile can prove when two sessions produce identical bytes for the same route — so the encode happens once and the bytes are shared.

Worth stating plainly: the three reload paths are not the same amount of work. SIGHUP re-parses only rustbgpd's policy files. birdc configure re-parses BIRD's entire config and re-evaluates its filters. bgpctl reload re-parses and soft-reconfigures OpenBGPD's RDE. Each is the thing an operator actually runs, which is why I measured it that way, but the architectural gap is smaller than 50x.

Deferred Registration Dumps

The flapstorm re-announce column didn't start where it is now. It started at a flat 9.5–9.8 s p50, about 2.5x slower than BIRD.

The distribution gave it away. p50 sat within 0.2 s of max, in every round of both runs, insensitive to round and run. Delivery jitter across 650 observers doesn't look like that — that shape means everyone is waiting on one shared completion event.

The mechanism: a reconnecting peer's PeerUp registration performed its initial full-table Adj-RIB-Out dump inline on the RIB actor loop. The 50 returning members' dumps serialized ahead of the survivor-facing re-announce fan-out already queued behind the burst. Head-of-line blocking, not a timer. Three things confirmed it before I touched any code: the plateau scaled with flap count × table size, survivors' first re-announce arrivals clustered right after the final dump finished, and the daemon log timeline showed the dump sequence occupying the actor for exactly the width of the plateau.

The fix: when the actor has queued work and the table is at or above 10k routes, a registration and its dump defer behind the queued imports, completing one per loop iteration strictly after each queued mutation batch drains. A quiet actor or a small table still registers inline. Graceful restart is untouched.

Re-announce p50 went to 0.49–0.55 s — about 20x — and the convergence and memory cells moved only within run-to-run spread, which is what a targeted fix should look like.

What the Gate Caught

The first attempt at every rustbgpd cell aborted against the campaign's 100 GiB tree-RSS gate, peaking around 104 GiB.

A just-shipped feature — RFC 7947 control communities honored at export — defaulted on for route-server clients. Its per-member export context disqualified all 700 sessions from update-group sharing, collapsing the route server into 700 independent full-table Adj-RIB-Out builds. Exactly the sharing described above, turned off by accident.

It never surfaced in unit tests or in any containerlab interop suite, because both layers are per-session correct and neither notices that sharing collapsed. The default flipped to opt-in the same night. This is the shape of bug a fleet-scale harness exists to catch, and it caught it before a release.

Speed Isn't the Reason to Switch

The benchmark is the part that's easy to put in a table. The reason I built this daemon is the control plane.

gRPC is the primary interface — eleven native services plus OpenConfig gNMI, and a thin CLI. Peer lifecycle, policy CRUD, route injection, streaming events, all without restarts. The config file bootstraps initial state and then gRPC owns the truth.

Policy is a typed, compiled language with named prefix and community sets, and unit tests that live in the policy file:

policy ixp-hygiene {
    # AS_SETs are deprecated (RFC 9774); reject them like arouteserver does.
    term reject-as-set { if route.as-path matches "\\{" { reject } }
    term reject-aspa-invalid { if route.aspa == invalid { reject } }
    term tag-ov-valid { if route.rpki == valid { add ext-community OV_VALID } }
}

test as-set-is-rejected {
    route { prefix 203.0.113.0/24; as-path "64501 {64502 64503}" }
    expect ixp-hygiene == reject
}

Then you can ask the daemon what would happen before committing anything:

rbgp policy check hygiene.rpol   # compile + run the in-file tests, offline
rbgp policy test hygiene.rpol --policy ixp-hygiene --direction import
                                 # read-only dry run against the LIVE RIB
rbgp rib --prefix 203.0.113.0/24 advertised 198.51.100.7 --explain
                                 # walk the real export gate to one peer

Every route decision explains itself from the live RIB — best-path, export-gate, and filtered-route views are always on, where incumbents need an external looking-glass stack for less. IXPs already running arouteserver can render config from their existing general.yml and clients.yml.

Where the Incumbents Win

BIRD wins memory. Settled RSS under flap churn is 337/328 MiB against rustbgpd's 441/451 MiB. At the reload shape it's a dead heat — 419/419 vs 422/412 MiB, one run each way — but the flapstorm advantage is clearly BIRD's, consistent with its reputation.

OpenBGPD wins the stall, by roughly 2x: 0.25–0.29 s against 0.42–0.60 s. Its RDE keeps churn flowing while it recomputes.

Two caveats on my own numbers. This is loopback TCP: no NIC, no RTT, no loss — syscall and PDU counts match production, but the network doesn't. And rustbgpd's stall drifts upward across consecutive reloads within a session, 0.43 → 0.68 s p50 over four reloads in one run, published per-reload rather than averaged away.

Try It in 60 Seconds

The demo spins up the daemon with an FRR peer advertising sample IPv4 and IPv6 prefixes. No real routers needed:

cd examples/docker-compose
docker compose up -d --build

docker compose exec rustbgpd rbgp -s http://127.0.0.1:50051 summary  # peer comes up
docker compose exec rustbgpd rbgp -s http://127.0.0.1:50051 rib      # browse the RIB
docker compose exec rustbgpd rbgp -s http://127.0.0.1:50051 top      # live TUI

Or pull the image directly: docker pull ghcr.io/lance0/rustbgpd:latest.

Every figure above links to its artifact set in the receipt matrix, with full methodology, the configuration disclosure for all three daemons, and reproduction instructions. The July campaign, every earlier band, and the raw artifact sets are all preserved there — including the pre-fix re-announce numbers above.

rustbgpd is public alpha, Linux only, MIT/Apache-2.0. If you run a route server or a reflector and any of this is interesting, I want the bug reports — rbgp.rs, GitHub.