std/json:parse: A Destructure Is a Schema

Koru already generates parsers at comptime — std/regex lowers each pattern branch to a specialized DFA function, and std/parser lowers a grammar to mutually recursive descent functions. std/json:parse is the next step in the same direction, and it does not take a grammar. It takes a destructure:

std/json:parse(input: body)
| ok { service: string, port: i64 } |> ...
| malformed { message, offset } |> ...

The ok branch’s destructure is the schema. The transform reads it at comptime and emits a bespoke recursive-descent parser for that exact shape: per-key string dispatch, a typed reader per field (string decodes escapes, i64/f64 convert at the splice, bool reads the literal), and the braces nest and qualify — a schema like { items: []{ id: i64, name: string }, tls: ?bool, limits: { rps: i64 } } names an array of objects, an optional field, and a nested object in one destructure, single line, no ceremony. Keys the schema does not name are skipped but still validated — malformed input routes to malformed with a byte offset, whether the damage is inside a field you read or one you skip. Duplicate keys are last-wins; a required field that never appears is malformed.

What comes out the other side is not a document — it is the bindings. There is no Doc, no root(), no object.get, no as.* conversions, and no close.

The trade it makes

The generated parser allocates what it must — decoded strings, arrays — from a per-site arena that is created when the ok branch begins and deinitialized when its chain exits. Field values are valid for the scope of the branch and die with it. That is a real limitation, and it is the whole trade: a parse result that must outlive the branch needs an owning document — which is exactly the thing std/json:parse declines to build. Koru already has a spell for that lifetime — *Doc<open!> with a checked close, the contract koru/yyjson carries — so the two surfaces divide the work rather than duplicate it: schema unknown at comptime, or a document that must be held, mutated, or re-serialized → yyjson; the program knows its envelope → std/json:parse.

The write side is the same shape

std/json:emit is the symmetric transform. The schema this time is the record literal at the call site — the output shape is written down where the program would have listed the fields anyway:

std/json:emit(value: { service, port, limits: { rps }, tls, weight }): body
|> std/json:emit.free(text: body)

The transform resolves each value’s type from the enclosing scope at comptime and generates a writer for exactly that shape: constant fragments go out as whole appendSlice runs, strings pass through a JSON escaper, numbers and bools write inline, nested records and arrays recurse. No builder tree, no document — bytes land in one buffer, toOwnedSlice hands back the text. The result carries an <emitted!> obligation: emit.free settles it explicitly, or scope exit settles it for you.

The measurement

Two races, both on the same protocol — 3-second timed passes in ReleaseFast, a checksum accumulation so nothing optimizes away — against two different kinds of rival. The first is the DOM: koru/yyjson, the owning document wrapper, racing the generated parser on what it refuses to build. The second is the honest same-work peer further down: serde deserializing into typed structs, the strongest schema-directed extraction on the market.

The question a DOM avoids asking is whether the DOM was worth building. The bench (koru-libs/yyjson/bench/bench.k) races four lanes over one ~250-byte service envelope. Two read lanes parse the document and read the same six fields — two scalars, a nested object field, an array length, a bool, a float — once per pass. Two round-trip lanes read a five-field subset, then write the same 83-byte object back out: std/json:emit against yyjson’s builder-plus-render. A grid cell accumulates a checksum so nothing is dead code, and a byte-count check confirms both writers produce the identical 83 bytes every pass. std/benchmarking:run drives each lane for a 3-second window in ReleaseFast — the only build mode anyone ships. Same binary, same document; M2 Pro. Numbers below are min of 2 runs.

The gap is ~20x on reads and ~21x on the round trip. The mechanism is visible in how it got there: the same bench in Debug gives the generated paths only ~2x, because yyjson’s arm barely moves between build modes — call overhead and the prebuilt C library dominate, and neither is Zig that was waiting to be optimized — while the generated code’s passes go up an order of magnitude the moment the optimizer gets a look at it. The DOM path was already as fast as its architecture lets it be; the specialized path was just getting started.

The write side is where the mechanism pays double. yyjson’s round trip builds a second tree — new.object, new.int, object.set per field — then walks it again to render; the generated writer appends {"service":" once and writes each value in place. Emit costs the schema lane about a quarter of its read throughput; the builder costs yyjson about the same fraction — but off a floor twenty times lower.

The same-work rival

The DOM race is the contrast, not the peer. serde + serde_derive deserializing into typed structs does the same work as std/json:parse — validate the whole document, bind declared typed fields, tolerate undeclared keys by skipping them (serde descends through IgnoredAny; the generated parser runs the same validating skipper). No Value, no DOM on either side. The schema is written down in both worlds — the difference is where:

#[derive(Deserialize)]
struct ItemMeta { k: i64, note: String }
#[derive(Deserialize)]
struct Item { id: i64, name: String, email: String, score: f64,
              active: bool, tags: Vec<String>, meta: ItemMeta }
#[derive(Deserialize)]
struct DocMeta { title: String, version: i64 }
#[derive(Deserialize)]
struct Doc { items: Vec<Item>, meta: DocMeta }

serde writes it as a struct forest in a declaration; Koru writes it once, at the point of use, in the shape of the bindings the branch is about to get — items: []{ id: i64, name: string, ... }. The koru-benchmarks json-parse suite races them in a schema-extract lane: same protocol as every contender (3-second timed passes), and a checksum gate that fails loud if either side’s bound fields sum to anything but the expected per-pass constant. Two modes:

Two honest results, not one. On the fully-typed document the generated parser binds at nearly twice serde’s rate — no struct machinery, no field name double-table, the dispatch is the shape. On the partial document the race is dominated by skipping the undeclared payload, and serde’s skip is very good — the margin shrinks to ~15%. Both contenders printed identical sinks on every rep; the lane’s gate would have killed the run otherwise. Laptop numbers, best of 3 — but the check that could falsify them ran inside the board.

Both gaps are structural, but different shapes of structural. Against yyjson the win is refusing to build the intermediate at all — the DOM answers field-at-a-time object.get/as.* calls on a tree the program never asked for. Against serde there is no intermediate to refuse; the margin is what the bound values cost — arena-scoped decoded strings and arrays against individually owned String/Vec allocations — plus dispatch specialized to the field set at comptime. None of it is free on our side either: the generated parser pays a fresh arena per pass and validates every skipped key. Laptop numbers throughout — measurements, not a protocol — but the shape of the result does not need statistical cover: in both races, one arm built no intermediate representation at all; against yyjson in either direction, and against serde on the one that matters.

What it settles

The lesson is not JSON-specific. Any input whose shape a program can name at comptime — a wire envelope, a config file, a protocol header — can get a bespoke parser instead of a DOM walk, and the schema is already written down where the program would have destructured the result anyway. The destructure is the schema; the branch is the lifetime; the parser is generated. When a benchmark’s workload has a known shape, this is the instrument that competes.

It is also a narrower instrument than the DOM. There is no mutation of a held document, nothing to pass across flows — values live and die inside a branch — and both schemas must be written down for the compiler to see. Where any of those are the requirement, yyjson is the answer — that is what it is for.

The collateral worth naming: exercising []string in a schema surfaced a compiler bug in lowerZigType — every []-prefixed string destructure type panicked the emitter’s @memcpy — and pushing to []{ ... } arrays of objects surfaced the deeper one: the schema’s destructure text was being spliced verbatim as a Zig type annotation, so items: []{ id: i64, ... } emitted []{id: i64, ...} — Koru schema text where a type belonged. The transform now rewrites each leaf’s type spelling to the generated __JP struct name before the generic emitter sees it; the schema the programmer writes stays Koru, the type the backend emits is Zig, and the boundary is deliberate.