The $12 Server Challenge: What the Real Box Refused

The benchmark is the twelve-dollar-server-challenge: a stock 1-vCPU, 2 GB, no-swap Ubuntu 24.04 droplet, k6 driving from a separate machine, and the score is simply the most connections held for a five-minute hold — p95 under 500 ms, p99 under 1 s, under 1% errors, or the run doesn’t count. No Nginx required; the server answers on :80 directly. The leaderboard leader is a Rust io_uring build at roughly 310,000 connections.

We set one rule up front: no submission unless first place is plausible. These benchmarks are worth a lot to us as oracles — the suite exercises the whole stack, Koru, Orisha, the standard library, SQLite, JWT, io_uring — and worth nothing as a public signal if we place second on a memory-scored board.

The local race

Correctness first: the challenge’s own suite, 42 of 42 checks, on every build variant we produced.

Then the fight that matters — memory per held connection, because held connections are the score. Against the leader’s published local baseline (170 MB at 20k VUs, p95 8.3 ms, p99 71.6 ms), the first Orisha builds were fast and fat: 626 MB at the same hold. The diet was a sequence of compile-time capability gates on ConnState — h2, websocket, send-plan, provided-buffer pool — plus shrink-on-idle, which releases the per-connection request and send buffers the moment a connection verifiably has nothing in flight.

Median stayed ~250 µs across every build. The p99 column is honest about its own noise floor: the local rig runs k6 and the server inside the same Docker VM, so the tail was rig-dominated — which is exactly what the dedicated box exists to measure.

At that point the projection said the build fits ~300k connections inside 2 GB where the leader runs tight. Worth a real machine.

What the box refused

Two identical-shape Hetzner VMs — a cpx12 (1 vCPU, 2 GB, the challenge shape) as server, a cpx22 as load generator, same datacenter. Native x86 build, direct on :80, SPEC environment, healthy /health.

Refusal one: provided-buffer rings are dead on stock Ubuntu 24.04. Every IORING_REGISTER_PBUF_RING returns EINVAL on 6.8.0-138-generic — any argument shape, any ring-setup flag combination, liburing or raw syscall. EVENTFD registration works; all the Ubuntu io_uring restriction sysctls are permissive. The kernel simply rejects the call. Since the challenge droplet is the same stock Ubuntu 24.04 x86 image, the leader’s io_uring build almost certainly hits the identical wall there — the fallback path is the fair battlefield, so we measured it rather than treating it as our bug.

Refusal two: the multishot acceptor retires at ~240 connections. On the one-shot receive path the kernel forces, the server accepted roughly 240 connections and then stopped — healthy on the ones it held, deaf to new ones. The accept loop re-arms the acceptor when F_MORE clears, and the re-arm’s catch {} swallowed the errno silently. The box is gone now — strace captured before teardown wasn’t retrieved — but the failure site is identified and instrumented, so the errno prints instead of vanishing the next time.

The probe also surfaced and fixed a real bug that had been invisible under the pool path: st.buf could be freed while an armed one-shot receive SQE still pointed into it — a use-after-free that passed the suite only because freed arena memory stays mapped. And the one-shot path’s eager 4 KB buffer per connection is now a 1 KB floor with grow-on-demand.

The fix, verified on the same kernel

The acceptor retirement turned out to need a small answer: when the kernel retires the multishot acceptor and refuses the re-arm, the worker flips to one accept SQE per connection — one extra submission per connect, still in-ring, versus a dead listener. The silent catch {} now logs every errno, and two env knobs (KORU_PUMP_NOPOOL, KORU_PUMP_PLAINACCEPT) let us drive both fallback paths on kernels that would never refuse them — which is how we proved the one-shot path itself was portable before ever renting a machine.

On a fresh cpx12, same stock Ubuntu 6.8 that capped at ~240: 43,843 connections held at ~130 MB RSS — about 3 KB per connection, comfortably inside the challenge’s 2 GB at the scale the leaderboard plays at. The 42-check suite stays green on the fallback build. Both refusals turned out to be the same posture — this kernel declines the io_uring resource-registration features the fast paths lean on — and it hits every io_uring submission equally, including the leader’s.

The honest accounting

• Fixed: capability-gated ConnState (~3× leaner per connection), shrink-on-idle (3.7 MB RSS after an idle drain on the first box; the 43.8k run retained ~166 MB in the arena — held peak vs. reclaimed memory, both real), the st.buf UAF, the small oneshot arm, the accept fallback with logged errnos.
• Still unposted: the official number. The accept path is proven at 43.8k held, but the k6 latency run needs a load box that can materialize 30k+ VUs — on the final attempt the cpx22 generator stalled at 29,424 of 30,000 VUs during init while the server sat at ~15k conns / ~102 MB mid-climb. There is no leaderboard claim in this post because there isn’t one to make.
• Environment note: local p99 is not adjudicable — 39 ms to 6.1 s across identical binaries in the shared-VM rig. Only the dedicated topology counts.

All boxes were deleted the same session; the bill was pocket change.

The submission, verified end-to-end

The submission now exists as a real folder — submissions/koru-orisha-denstad/ — and we ran its own CI path on a fresh cpx12 before anyone else’s automation ever saw it: install.sh as root (deps + the zig 0.15.2 toolchain), build.sh as a normal user (three pinned-hash clones — koru 01b2d5c9b, orisha 6d8502682, koru-libs 9a60d33 — building the compiler then the app), and the challenge’s own test/run.sh orchestrating start-on-fresh-seed plus the 42-check suite. 42/42 on real x86 Ubuntu 24.04, the binary on :80 in direct mode. The only failure the whole pass produced was a jq PATH hiccup in our own harness — the server itself never missed a check.

What it still isn’t: a leaderboard number. The submission is verified correct and verified lean; whether it wins the five-minute hold at the leader’s scale is the one question the challenge’s own boxes answer and ours haven’t yet.

What happens next

Open the PR — now filed — let their CI run it, and — separately, because it’s the honest measure — one bigger load box to answer the C100k+ latency question for ourselves before the leaderboard does. If that number confirms the local race, the submission stands. If it doesn’t, this post, the fixes, and the std/inject design the work produced are what the benchmark was actually for.

The capability mechanism itself is slated to become a first-class language concept — [inject]mod:tor|variant declared in a foreign file, reusing the compiler’s existing cross-module impl and variant machinery. The design is parked as a concept frag in the koru repo; it changes where the lean code lives, never what it compiles to.