Benchmarks: Koru on the Kostya Suite, the LangArena Board, and the Goose Race

Koru’s first entry on Kostya’s language suite is in the queue — five tasks (base64, brainfuck, json, matmul, primes), each a pure-.k driver with one .kz shim for the suite’s notify socket, open as PR #521 from the korulang fork. It isn’t Koru’s first external board: Orisha, the Koru HTTP server, is already merged on HttpArena — its io_uring entry sits in the top ten on the baseline test at ~4.1M requests/s, a number the HttpArena maintainer’s own harness produced. Alongside the Kostya submission sit two private boards: the 47-task LangArena suite replayed on macOS and a Linux droplet, and a five-task race against Goose on both architectures.

Every number in this post is from our own runs — this laptop (Apple Silicon, macOS) and a rented shared 2-vCPU Debian droplet, since destroyed. The suite’s published table is produced by the maintainer’s own semi-manual runs inside the pinned Docker image on a Xeon E-2324G; nothing we ran feeds it, and no number below should be read as a prediction of it. What our runs establish is that the protocol works end to end and where Koru stands on boards we control.

The protocol decides what gets timed

Kostya’s suite times a program between two TCP messages: an announce (send-pid) opens the scored window, a stop closes it. Everything between them is measured; everything outside is not. That makes the bracket the whole game — a driver that verifies its own correctness before announcing can never bank a time on a program that doesn’t work:

// verify (find(100,2) sorted must be exactly [2,23,29]), then announce,
// then the scored find with its sort inside the bracket.
find(lim: 100, pfx: 2): _ |> sortres(): _
|> if(st.nres != 3 or res[0].v != 2 or res[1].v != 23 or res[2].v != 29)
    | then |> std/io:print.ln("VERIFY_FAIL")
    | else |> notify:send-pid(name: "Koru", port: 9001): _
        |> find(lim: 5000000, pfx: 32338): _
        |> sortres(): _
        |> notify:send(msg: "stop", port: 9001): _
        |> show()

This is the primes driver end to end: a small query through the real sieve+trie+walk path must produce exactly [2, 23, 29], then the process announces, then the scored find(5000000, 32338) runs with its result sort inside the window because the reference sorts inside find. All five drivers carry this shape — self-check, announce, scored work, stop — and all five wrote results.log rows under the real unpatched xtime.rb on a Debian droplet: base64 0.602s, brainfuck 2.206s, json 0.311s, matmul 4.233s (n=1500), primes 0.121s. Droplet context numbers, not standings — the machine is a shared 2-vCPU box.

Each header also states what it is not: matmul discloses static flat grids (BOUND_FAIL past n=1500) and an i-k-j row-streamed multiply instead of transpose+dot — bit-identical accumulation, verified against an independent C oracle to 13 significant digits. Base64 discloses select chains where the Rust and Zig entries use SIMD. Brainfuck’s announce deliberately precedes the file read, making Koru’s window a superset of the reference’s rather than a subset.

The private boards

The bigger sweep ran on LangArena — 47 tasks, verify-first (every output diffed against expected.txt before any timing), three interleaved repetitions, min-of-N. The basis is deliberately asymmetric: Koru is timed whole-process, startup and warmup included, while the Rust max-perf reference is timed on its internal Instant window alone. Whatever the board says, it understates Koru rather than flatters it:

Geomean here is exp(mean(log(ratio))) — every task weighted equally, a 2× win and a 2× loss cancelling exactly. The rows that stay behind are the short I/O-bound ones (base64, logparser, bwt-decode) where process startup weighs heaviest inside our side of the asymmetry. Bucketed by speedup the shape is the same on both machines — the laptop run: 5 tasks slower, 19 inside parity–2×, 12 at 2–5×, 5 at 5–10×, 5 at 10–50×, 1 past 50×; the droplet: 4 slower, 9 inside parity–2×, 16 at 2–5×, 13 at 5–10×, 4 at 10–50×, 1 past 50×.

Goose — Wouter van Oortmerssen’s memory-safe, heap-free language — got a symmetric race instead: both sides whole-process, interleaved min-of-5, verify-first, same commits on both platforms. Goose’s own Kostya submission is in the same queue ours just joined (PR #520, from the language author’s fork). The ratios transfer:

What the suite made the compiler do

The language grammar did not move for any of this — the week’s compiler work was integration and honesty, not features. koruc -o names the output binary; KORU_ZIG names the toolchain version explicitly, which matters because the suite pins Zig 0.16.0 for its own tasks while koruc wants 0.15.2 — one env var, all nine spawn sites in both processes honoring it, a wrong path failing loud instead of silently falling back. parse-int widened to string<*> so argv parses without a copy. And the diagnostics got less forgiving: an unspaced a-b is an identifier by design, and the compiler now says so instead of emitting the program you meant.

The toolchain claims sit on pinned tests — the suite entry uses std/args for argv, and the compiler refuses a target it doesn’t know rather than guessing.

The kernel probe — n-body as the language’s own vocabulary

One LangArena driver got rewritten as what it actually is — and the rewrite fixed the kernel. clbg_nbody was a 262-line hand-tuned grid kernel; it is now 142 lines spelled shape / init / step(0..24000000) / pairwise / self / reduce / computed — checksum-identical (338799744) at ~0.64s against the tuned driver’s ~0.62s on the same box, inside the noise band. Both energies are computed, none pinned: the kernel folds kinetic and potential energy over the initial state, runs 24M integration steps, folds them again over the final state, and one | reduce r |> exit carries r.v1 and r.v2. The ordering worry that kept the original verbose — the checksum pins accumulation order — is honored directly: the folds are strict-float self/pairwise bodies accumulating in the reference’s exact traversal order.

The probe’s real yield was three defects in std/kernel, all fixed upstream and regression-pinned: a reduce written before step was silently absorbed into the step loop and accumulated per step (measured: a total of 15 where 3 was correct) — kernel ops now carry their position in the |> chain and emit pre/inside/post accordingly; a | reduce |> exit collected only the first reduce op’s accumulators — it now merges every op’s, which is what makes the measure-evolve-measure spelling possible; and a second std/kernel:step silently overwrote the first’s range — KORU127 refuses it. One performance defect fell too: the pairwise double loop emitted for where while measured ~5% faster on the real emitted artifact — the codebase’s emit-for-not-while invariant was amended to admit measured exceptions rather than bypassed. The remaining edges are recorded in PARKED.md (kernel:init stays single-exit for computed arrays; the GPU and FPGA surfaces refused this kernel by design — f64 and step respectively).

The expressiveness column

LangArena also scores each language on source size — bytes, gzip bytes, and line count folded into an expressiveness rating. Here the Koru corpus carried a tax nobody else pays: every other language on the board ships with zero comment lines — not few, zero — while a third of Koru’s was the per-driver disclosure headers (3,882 lines) stating what each implementation does differently. So the submitted sources are generated: the canonical tree stays annotated, and a strip script emits the artifact the suite scores — rebuilt and output-verified identical before staging. Code-only, the density is mid-pack: 154 lines per task against Go’s 153, Rust’s 140, Zig’s 187, Nim’s 88, Goose’s 102.

The submitted diff is eighteen files — five drivers, five shim copies, five Makefile hooks, the shared KORU_BUILD recipe, the Dockerfile toolchain stanza, a versions-table row — public on the korulang/benchmarks fork and awaiting review in the suite’s queue. The suite merges in maintenance sweeps every few months, so the official table updates on the maintainer’s clock; what the post’s numbers establish — protocol conformance end to end, and standings on boards we control — is already measured.