The pump composes verbs, not stores

· 10 min read

std/pump is out. Two independently blocking systems — vaxis’s terminal event loop and orisha’s socket server — now compose at compile time into one cooperative driver, share a single kernel poll() over the union of their interests, and drain cleanly together. The contract a participant answers is three verbs and zero branches, and in vaxis’s case the abstraction went all the way down: vaxis:run is no longer a hand-written loop at all — it is a transform that emits the pump itself.

This post is the contract first, the proof second, and the road that got ruled out last — the rejected designs and what digging past them found in the compiler.

What the pump composes

vaxis’s terminal loop (koru-libs/vaxis/index.kz) and orisha’s socket loop (orisha/lib/pump.k) are the actual things std/pump exists to compose. Their contracts, verbatim:

pub tor run { port: u16, tls_port: ?u16, cert_path: ?string, key_path: ?string }
! arrived *Exchange
| stopped string
| failed string

orisha: one effect (! arrived per request, resuming — the loop continues after the arm returns) and two terminals — | stopped, | failed — the only ways out.

vaxis is the same shape scaled up: forty-odd optional effect arms — ! draw, ! char, ! key-raw, ! tick, ! paste, the mouse family — and exactly two terminals:

! ?tick i64
| ?done
| err string

Look at what is not there. No | event. No | no-event. No per-iteration answer at all. Inside the body, the loop asks the OS directly — $mod.event_loop.tryEvent() returns ?Event, a value — and “an event happened” is that the optional isn’t null. Everything the loop emits mid-flight is an effect. | is reserved for “the run is over” — done, err, stopped, failed. In both pumps, that invariant holds completely. The contract below is the vocabulary those two loops already speak.

The contract: three verbs, zero branches

A participant is anything that answers up to three verbs — plain calls, compiled per join site into pump-local units:

  • step() -> i32 — advance every live instance once; return how many progressed. Required. Progress is a return value.
  • live() -> i64 — how many instances are live; 0 = retired. Absent = the participant is immortal, and a pump of immortals runs forever — honest, because that pump is the program’s event loop.
  • wait(i) -> { fd: i32, wait_ns: i128 } — instance i’s blocking interest, enumerated 0..live(). Optional; feeds the union wait.

No | anywhere in the contract. The join site is the reference form — the same shape std/store(name) ! field uses for standing watches:

std/pump:create(main)
| drained |> std/io:print.ln("all participants retired")

std/pump(main)
! step |> tick-step()
! live |> tick-live()
! wait i |> tick-wait(i)

std/pump(main)
! step |> tty-step()
! live |> tty-live()
! wait i |> tty-wait(i)

std/pump:run(main)

Pass order is join order — document order. run loops passes until every live answers 0, then fires | drained. A dead pass fires ! idle if armed; unarmed, the pump collects every joined wait interest and blocks once on poll(), capped by the nearest wait_ns.

A store is a participant, not the participant

A ! step/! wait store exposes the three verbs as ordinary generated units — tick-step steps every live row once (retiring | complete in place, counting progress as i32), tick-live is the row count, tick-wait(i) reads row i’s interest. Inside the store, | complete still earns its | — it genuinely retires a row. | stays where it means over, one level down, and the pump never sees it.

And the singleton — the case a store-shaped pump could never express — joins identically. probes/r7 is a module holding raw state (var count, var budget) with three host tors:

std/pump(main)
! step |> lib/single:step()
    ! stepped c |> std/io:print.ln("single stepped {{ c:d }}")
! live |> lib/single:live()
! wait i |> lib/single:wait(i)

It runs, it drains, and the join is indistinguishable from the store’s — which is the whole point. vaxis’s tryEvent()-shaped loop and orisha’s connection set can both sit behind this contract without either one pretending to be a container.

The proof: two real pumps, one driver

orisha/examples/pump-tui joins orisha’s server and a vaxis TUI under one std/pump(main) — the two real pumps from the top of this post, sharing a single poll(). Orisha answers GET / while vaxis owns the alternate screen; one q keypress retires vaxis’s live, orisha’s stop follows, both deinitialize, and the process exits 0. The listener fd and vaxis’s wake pipe (the fd-ified stand-in for a condvar queue that cannot be polled) sit in the same kernel wait. No second loop, no thread doing the other system’s work.

The part that surprised me: making it true required deleting vaxis’s own driver. vaxis:run was a ~250-line proc hand-spelling exactly the init/step/live/wait/deinit middle the participant verbs already carried — duplicated because the proc could not forward the call’s ! arms. Effect splicing gives a proc __H, the handler struct — usable once, at its own call site, and never as a value. step_event.handler(.{}, __H) is unspellable. The composition a proc cannot express, a generated call site expresses trivially: vaxis:run is now a transform that emits vaxis:step() <your ! arms> as a generated unit — the arms splice there exactly as they do for authored code — plus a generated driver that polls live/wait and calls deinit. The duplicated middle is gone; run literally lowers to the pump. A hand-written loop carrying a second copy of dispatch “because the arms cannot be passed on” is a tell: that loop wants to be a generated call.

What it settles

The vocabulary now matches the two real pumps exactly: ! = something happened inside a pass, -> = what a verb found, | = a run is over — and the pump’s only | is | drained, which is exactly that. Composition is compile-time: the joins are sites, the units are generated, the loop is emitted — and now, in vaxis’s case, so is the run that used to fake it by hand. State, plurality, and lifecycle stay inside the participant, where the real pumps already keep them.


The road that got ruled out

Everything above is the destination. This is how it went wrong first — kept because the two failures are the clearest explanation of why the contract has this shape, and because the journey is half of what these posts are for.

What I built first

The first version coupled two mistakes. A pump participant was a std/storestd/pump:new(main) { tick tty sock } named stores, and the pump discovered generated store units (__store_step_<s>, __store_waitone_<s>) and iterated their rows. And the row’s step answered with verdict branches: | running for progress, | complete for retirement, any other word — | quiet, | stalled — for no-progress.

It worked. probes/r6/target.k ran it on real socketpair fds — two fd sources and a deadline tick, one thread, one kernel poll() over the union of row interests. Green tests, honest waits. The shape was the problem.

Objection one: | quiet is not an exit

| is a continuation branch. It means the call exits. ”| quiet” — nothing happened, ask me again — is a progress report wearing exit syntax.

Steel-manned, the step call does technically terminate each pass, so | is mechanically legal. But the vocabulary trains readers wrong. | stopped, | failed, | done, | err all mean the run is over; | running in that slot looks like the loop ends by running. A dead poll is not a terminal state — it is the absence of an event — and the two real pumps put absence in a return value or in silence, never in a branch.

I renamed it | quiet and kept going. That answer was wrong too, and objection two showed why.

Objection two: the store is not the participant

A store is a runtime container. A pump is a compile-time schedule. Any trace of std/store in std/pump is the dependency inversion: composition happens at transform time, and forcing every participant through a plural-row container universalizes the one case that happens to be plural.

Steel-manned, the store was convenient scaffolding — orisha’s connection set genuinely is plural, and rows gave per-instance retirement for free. But vaxis’s event loop is a singleton: one loop, module state, no rows. Under the store-coupled pump, it would have joined as a one-row store wearing a wrapper — the abstraction forcing every participant into the plural case. The pump knew store internals (generated unit names, column fields, .len) — which meant a participant couldn’t be anything else.

The re-slice deleted both mistakes at once: progress moved into a -> i32 return, and composition moved from “stores the pump iterates” to “verbs the pump names.” Everything in the contract section is what survived.

What the pump dug out of the compiler

The pump’s other job was being heavy enough to break the toolchain honestly — it is a compile-time composition instrument, and it found four bugs in one session:

  • A fixed [32] effect-site buffer overflowed once a transform pass held more than 32 sites — allocator-sized now.
  • Dead stripping skipped proc bodies for sibling_event.handler references — generated pump drivers lost the very handlers they called.
  • A qualified type resolved its home through a bare-name map. ?std.mem.Allocator reduced to Allocator, which a private convenience alias inside [comptime] std.compiler had claimed — so every consumer naming an allocator welded the entire compiler module (AST, passes, logging) into its runtime binary. Qualified spellings now resolve their qualifier; the map is the bare-name fallback. This one is my favorite kind of bug: invisible until a real consumer, catastrophic in exactly the place it should never reach.
  • @import("root") references are invisible to reachability — the OpenSSL decls orisha’s generated code calls were being stripped; the public tors carry [retain] now.

None of these are pump bugs. They are compiler bugs the pump was the first instrument wide enough to trip over — the reason the library exists at all is to surface them. I was wrong twice getting here; this is what the detour bought.