Kernel Fusion: One Pass, Not Several

The kernel’s whole premise is that you describe relationships, not iteration, and the compiler decides the loop. You write self, pairwise, reduce, and the language is supposed to hand back the tight thing. Working on reduce this month, I came to think the promise is only half-true — and that the honest story isn’t about speed, it’s about fusion.

Here’s what you actually write. A kernel describes relationships, not loops: self advances an element, reduce answers a question about the state, and the compiler is supposed to pick the loop.

import std/kernel
import std/io

std/kernel:shape(Pt) {
    x: f64,
    mass: f64,
}

std/kernel:init(Pt) {
    { x: 0.0, mass: 1.0 },
    { x: 1.0, mass: 2.0 },
}
| kernel k |> std/kernel:self { k.x += 1.0 }
    |> std/kernel:reduce {
        total_x[f64]
        count[i32]
        total_x += k.x
        count += 1
    }
| reduce r |> std/io:print.blk {
        total={{ r.total_x:f }} count={{ r.count:d }}
    }

self turns x = [0, 1] into [1, 2], then reduce totals it: total=3 count=2. No iteration spelled anywhere — that’s the whole pitch.

std/kernel:reduce lives inside the kernel scope, but a kernel:init must hand its caller an exit branch. It has two choices: | computed c |> binds the mutated array (c[i] is element access), or | reduce r |> binds a scalar aggregate. A scalar has nowhere to live in an array-shaped exit, so reduce carries its own branch — not by accident, but because it is the matching exit for its result shape.

One thing to notice: total_x and count aren’t fields of Pt, and they’re not declared elsewhere in the flow — because the reduce block DECLARES them itself. A name[type] line in the reduce header is a scoped slot: the accumulator is declared (with its type) in exactly the block that introduces it, the transform zero-initializes it and carries it by name through the | reduce r |> payload, so r.total_x reads it. Nothing is inferred from usage — count is i32 because you said so, and the shape says what the data is while the reduce block says what it accumulates and under what names.

And notice what that means about scope. A name declared inside a Source-block reaches out and binds onto the enclosing continuation: total_x lives in the reduce block, but you read it as r.total_x outside. That is unconventional — an inner block minting a binding for the outer scope isn’t how most languages behave. It isn’t magic, though, and the difference from the magic we replaced is the whole point: you declared it. The declaration is exactly what says “this block’s output has a field named this.” The reduce block isn’t a closure computing into a void — it’s the event’s body, and its declared slots are the payload it hands back. Every mini-DSL that introduces a name follows the same rule: declare it, and the transform decides whether it scopes locally or rides the handle.

Reduce is genuinely faster than the C people write

std/kernel:reduce accumulates a scalar out of a kernel scope. Measured against a C reduction over the same changing data, on a clean machine:

Koru is ~5x faster than the C people actually write — and it matches the ceiling a C programmer can only reach by flipping a global unsafe flag or hand-writing partial-sum accumulators. That isn’t a miracle. It’s a contract: reduce is a fold, so it has no order-dependence, so the compiler is allowed to reassociate it — vectorize on changing data, fold on static. Default C can’t do this without global fast-math (which breaks exact arithmetic). Koru grants it to one op.

The wall: the fold hides the loop

The interesting part is what happened when I tried to prove the win. Every time, the data kept constant-folding — an arithmetic progression, a static array re-run the same way each pass — and the measured time collapsed to near nothing. To see anything real, I had to engineer data that refuses to fold, so LLVM has to actually walk the data and do the work.

That resistance is the tell. The fold is an algebraic curiosity that hides the real signal. The real signal is the loop — and once you stop the fold from obscuring it, you see the loop is the thing we’ve been ignoring.

The kernel doesn’t fuse

A kernel that mutates and then aggregates under a step emits two loops — one for self, one for reduce (this is the real emitted structure, abbreviating the fold-resistant mass update):

for (0..n) |_step_i| {
    _ = &_step_i;
    for (0..64) |i| {
        __koru_fused_ptr[i].mass += __koru_fused_ptr[i].mass * 1.0e-12;
    }
    @setFloatMode(.optimized);
    for (0..64) |i| {
        total_mass += __koru_fused_ptr[i].mass; count += 1;
    }
    @setFloatMode(.strict);
}

Two separate walks over the array per pass. The kernel transform emits a loop per op and lets LLVM optimize each one alone. This is a documented, green state — the 390_060 test’s own header says it outright: “today these emit separate code blocks (two ptr extractions, two loop nests). Future: init should fuse the kernel branch into one block with one ptr extraction.”

So “the compiler decides the iteration strategy” is half-delivered. It decides which loop; it doesn’t decide one loop.

Fusion is the contract’s real prize

Here’s the thing the reduce work was circling. You can only fuse a reduce with a mutating op if the reduction is allowed to reassociate — because a single fused pass reads and accumulates each element while it mutates it, which relocates the sum relative to everything else. The strict contract forbids it. The associative contract permits it.

So the associativity isn’t a standalone win. It’s the prerequisite. It’s what makes the following legal:

for (0..n) |_step_i| {
    _ = &_step_i;
    @setFloatMode(.optimized);
    for (0..64) |i| {
        __koru_fused_ptr[i].mass += __koru_fused_ptr[i].mass * 1.0e-12;
        total_mass += __koru_fused_ptr[i].mass; count += 1;
    }
    @setFloatMode(.strict);
}

One pass. Mutate and accumulate in the same walk. That’s the kernel keeping its promise — and it’s the same contract shape that made reduce beat default C. The reason we kept fighting folds is that the fold was hiding the loop, and the loop is where fusion happens.

The honest caveat

Fusion is a memory win, and it pays when the array is bigger than cache and the workload becomes memory-bound. On our current corpus — a few dozen bodies that sit in L1 — walking the array twice is within noise. So the corpus, as it stands, can’t see the kernel’s best self. That’s part of the problem too: the benchmark suite exercises exactly the size where the kernel’s structural advantage is invisible.

A contract that says what an op may do

The pattern across all of this is one idea: a kernel op is defined by a contract that says what it may do — associate, be fused, stay exact — and that contract is what entitles the compiler to be bold. reduce may reassociate; self and pairwise may not, because they feed an oracle that has to be byte-identical. Declare the right contracts and the language gets the fast path without a global unsafe flag, scoped to exactly where it’s safe. reduce was the first such contract, and it is on-par-or-better than real-world C. The next one — fusion — is where the kernel’s actual promise lives, and it is a library change away.

The self+reduce compose-under-step shape — the first target where a fused kernel would show its worth — is exercised by 390_112_reduce_self_under_step, which keeps the two-loop emitted structure green while fusion is still ahead of us.