Durable Cells and the Fleet: Programs Over the Wire

A program that talks to a named thing — the assistant that remembers its sessions, the room that holds its occupants, the counter that survives restarts — needs a unit that is not quite an actor and not quite an object: an identity that owns state, takes turns one at a time, and is allowed to fall out of memory without dying. Cloudflare’s Durable Objects, Deno’s celld, and Orleans’ grains all land on the same shape.

Koru’s version is three standard-library modules, stacked: std/cell, the named state owner. std/node, the cache that keeps residents over a durable directory. And std/fleet, the protocol for the several nodes that share one world — ownership records, epochs, leases, and a mailbox for the callers they refuse. None of it is a runtime you opt into; it’s library code over the same obligation and continuation machinery every other module uses.

A cell is a shape, an id, and a session

A cell’s shape is a std/proto declaration — the name is resolved at compile time, so open cannot spell a shape nobody declared:

import std/io
import std/cell
import std/proto
import std/runtime
import std/interpreter

std/proto(room_state) {
    visits: u32
}

pub tor greet { name: string } -> string
greet -> name

std/runtime:register(scope: "cell") {
    greet(5)
}

std/cell:open(room_state, id: "assistant")
| cell c |> std/cell:run(c, source: "greet(name: \"assistant\")")
  | result r |> std/interpreter:value.stringify(r.value): j |> std/io:print.ln("cell said: {{ j:s }}")
  | error e |> std/io:print.ln("turn failed: {{ e.kind:s }}")
  |> std/cell:hibernate(c) |> std/io:print.ln("assistant hibernated")

cell said: {"branch":"","value":"assistant"}
assistant hibernated

Three things are doing the work here. The | cell arm hands the caller the cell! obligation — owned custody of the cell, settled only by hibernate, which wants <!cell>. The run step is a turn dispatched through the cell’s own interpreter against its registered scope — the grain method call — and its outcome vocabulary is two-tiered: | result when the turn completes, | error carrying every failure class as one record (e.kind names the variant). The result’s r.value is the turn’s structured value — {branch, fields} rendered as JSON by value.stringify, so the envelope in the output is the real shape crossing the wire; greet echoes its name argument, which is what proves the parameter made the trip. A caller that distinguishes classes sends e through std/cell:error-decode, whose optional arms fire per kind with the typed payload intact — p.line on parse-error, d.event_name on dispatch-error. And hibernate lives inside the | cell arm body because an arm’s bindings end with that arm.

The transaction has a lexical edge

State changes go through capture — and the commit boundary is not a call you make, it is where a lexical extent ends:

std/cell:open(room_state, id: "room-7")
| cell c |> std/cell:capture(c, shape: room_state)
    ! as s |> std/io:print.ln("seeded visits={{ s.visits:d }}")
      |> std/cell:captured { visits: s.visits + 1 }
      |> std/cell:captured { name: "assistant-01", awake: true }
    | captured f |> std/io:print.ln("c1 captured: visits={{ f.visits:d }} name={{ f.name:s }} awake={{ f.awake:any }}")

seeded visits=0
c1 captured: visits=1 name=assistant-01 awake=true

! as s seeds a live record typed from the proto — s.visits is a u32 field, not a map lookup. std/cell:captured { f: expr, g: expr } blocks dissolve into field-wise assignments on that record — one block, many fields. Chained captured statements do not commit as they go: they accumulate against the record and the whole set writes back in one seam when the ! as arm’s extent ends — the same moment Orleans’ WriteStateAsync marks, expressed as grammar. | captured f is the after-read of the committed record; a second capture on the same cell seeds from committed state. The flush is the single point a real materializer — SQLite, an object store — will eventually own; today’s writes state.kv into the cell’s world directory.

The guards are compile-time: a write to a field the shape never declared refuses naming both field and shape, a capture without its ! as arm refuses, and the bare ambient captured {} — control.kz’s vocabulary, which this surface deliberately does not reuse — refuses with a pointer to std/cell:captured. A caller that wants a write-only capture binds _ instead of a name and the transform mints the record anonymously.

A node is the custody boundary

Cells inside a std/node are never owned by the caller — the caller only ever holds borrows, and the std/node(expr) transform reads the expression’s shape to decide what kind:

std/node:new(node, scope: "cell", world: "", capacity: 8)

std/node(node)
| node f |> std/node:spawn(f, id: "room-42", cell: room_state)
  | spawned b |> std/io:print.ln("spawned room-42")
  | resident |> std/io:print.ln("resident")
  | refused |> std/io:print.ln("refused")

std/node(node) borrows the node; std/node(node.room-42) borrows the resident cell under that id; std/node(node["id"]) is the same with a runtime key. spawn’s cell: argument is the same comptime proto resolution as open — an unknown shape refuses, and omitting it refuses with a node member is a cell — name the shape.

The borrow carries *Cell<cell> — read-only custody. run and capture compose on it, but hibernate wants <!cell>, which lives in the node’s index: asking to hibernate a borrowed cell is a KORU030 phantom mismatch at compile time, not a runtime footgun. Custody is single-process, and the compiler is the enforcer.

Residency is a cache; the directory is the truth

Commit, evict, re-activate, retire — the whole lifecycle:

std/node(node.room-42)
| cell c |> std/cell:capture(c, shape: room_state)
  ! as s |> std/cell:captured { visits: s.visits + 41 }
  | captured f |> std/io:print.ln("committed visits={{ f.visits:d }}")
| not-found |> std/io:print.ln("lost")
std/node(node)
| node f |> std/node:evict(f, id: "room-42")
  | evicted |> std/io:print.ln("evicted")
  | absent |> std/io:print.ln("absent")
std/node(node.room-42)
| cell c |> std/cell:get(c, field: "visits")
  | value v |> std/io:print.ln("reactivated visits={{ v.int:d }}")
  | unset u |> std/io:print.ln("state lost")
  | unknown-field e |> std/io:print.ln("no field")
| not-found |> std/io:print.ln("reactivation failed")

committed visits=41
evicted
reactivated visits=41

evict flushes state.kv into <world>/<id>/ and frees the resident; a later member lookup that misses memory probes the floor, reads the shape header, and re-materializes the cell with state intact — Orleans’ activate-on-demand against a flat file. not-found now means never existed, not “not in memory.” spawn on a dormant-but-durable id reports resident — the identity is taken, not re-minted. And retire is the durable counterpart of evict: the directory dies, and not-found means gone for real. The leak checker counts every byte the registry holds — a program that doesn’t drain fails loudly.

A fleet is a protocol over the shared world

Two nodes pointed at one world directory need a shared answer to who owns this id — and a consequence for being wrong. std/fleet is that layer: a record directory under .fleet/, where claims, epochs, leases, and mailboxes live as files every node can read.

std/fleet:claim(world: "world/", id: "room-42", node: "node-a", ttl-ms: 0)
  | claimed e |> std/io:print.ln("node-a claimed epoch={{ e:d }}")
  | held-by o |> std/io:print.ln("a refused: held by {{ o:s }}")
  | failed e |> std/io:print.ln("claim failed {{ e:s }}")

std/fleet:seize(world: "world/", id: "room-42", node: "node-b", ttl-ms: 0)
  | seized e |> std/io:print.ln("node-b seized epoch={{ e:d }}")
  | free |> std/io:print.ln("no owner to take from")
  | contended |> std/io:print.ln("seize contended")
  | failed e |> std/io:print.ln("seize failed {{ e:s }}")

std/fleet:guard(world: "world/", id: "room-42", node: "node-a", epoch: 1)
  | owner |> std/io:print.ln("node-a still owns")
  | fenced e |> std/io:print.ln("node-a fenced: record now epoch={{ e:d }}")
  | free |> std/io:print.ln("no owner record")

node-a claimed epoch=1
node-b seized epoch=2
node-a fenced: record now epoch=2

This is celld’s core invariant — the conditional write that fences a stale owner — as a runtime vocabulary rather than a storage-driver detail. claim mints an ownership record and an epoch; seize takes the id and bumps it; guard is the check a holder performs before it writes: a guard on a superseded epoch answers fenced and names the epoch now on record, so an in-flight write from the old owner becomes a refusal, not a silent overwrite. Leases carry ttl-ms — an owner’s claim expires on schedule, renew extends it, and a claim against a live lease answers held-by naming the holder rather than seizing behind its back.

Ownership is also enforced where it matters — at the node. spawn and member lookups on a foreign-owned id answer held-by naming the owner; a dormant-but-owned cell re-activates only into the node of record; a capture write through a stale custody is fenced at the epoch check.

The mailbox: held-by is not a dead end

A refused caller in most grain runtimes gets an exception and a retry loop. Here it gets a mailbox. std/fleet:send posts a turn to .fleet/inbox/<id> — the box belongs to the cell, not to whichever node happens to hold it, so mail posted while the id is unowned waits for the next owner. The owner of record drains it under the lock; each mail is a real turn against the resident cell; the file is deleted only after its ! mail arm returns, which makes delivery at-least-once — a crash mid-turn redelivers on the next drain.

std/node(beta.room-mb)
| cell c |> std/io:print.ln("beta activated (BUG)")
| held-by o |> std/fleet:send(world: "world/", id: "room-mb", from: "beta", body: "touch(k: 5)")
    | posted |> std/io:print.ln("beta posted mail to {{ o:s }}")
    | failed e |> std/io:print.ln("send failed {{ e:s }}")
| not-found |> std/io:print.ln("beta: not-found")

std/fleet:drain(world: "world/", id: "room-mb", node: "alpha")
! mail m |> std/io:print.ln("mail from {{ m.from:s }}: {{ m.body:s }}")
  |> std/node(alpha.room-mb)
    | cell c |> std/cell:run(c, source: m.body)
      | result r |> std/cell:capture(c, shape: room_state)
        ! as s |> std/cell:captured { hits: s.hits + 1 }
      | error e |> std/io:print.ln("turn failed: {{ e.kind:s }}")
    | held-by o |> std/io:print.ln("alpha lost room-mb mid-drain: {{ o:s }}")
    | not-found |> std/io:print.ln("room-mb vanished")
| done n |> std/io:print.ln("alpha delivered {{ n:d }} mails")
| held-by o |> std/io:print.ln("alpha drain refused: {{ o:s }}")
| free |> std/io:print.ln("room-mb unowned")
| contended |> std/io:print.ln("drain contended")

alpha claimed epoch=1
beta posted mail 1 to alpha
beta posted mail 2
mail from beta: touch(k: 5)
mail from beta: touch(k: 2)
alpha delivered 2 mails

A stale node’s drain answers held-by and consumes nothing; a drain on an id nobody owns answers free — the mailbox never executes mail against an owner that isn’t.

The wire carries programs, not invocations

Look at what send takes: body: "touch(k: 5)" — source text — and what drain does with it: run(c, source: m.body), the same interpreter a REPL turn goes through. The routed turn is a program. That is a different wire protocol than the grain family it sits next to: celld’s wire carries names — a deployed handler, invoked by name-plus-arguments over HTTP — and Orleans’ wire carries method calls on a deployed grain interface. Koru’s wire carries the computation itself.

Two things follow. The first is no deploy boundary: a caller composes the turn at send time — the cell needs nothing shipped ahead of it, no handler deployed, no version to agree on. The second is atomicity: the body isn’t limited to one call. A turn is a multi-item program — invocations sequenced in one source text, bindings shared between steps, ?partial reporting ran/total if it stops early (the agent-channel grammar, already live in the bridge — the model’s reply is a program, not a message, and the same wire serves a mail file). A read-modify-write that conditionally commits — “increment if under N, else no-op” — is one mail and one commit seam, where the invocation-wire spelling is two calls with a decision and a race between them.

The honest price is ~79µs per turn — the interpreter dispatch, isolated by running turns with no materializer at all. Koru pays interpret-per-turn where celld pays deploy-once-then-invoke; a mail body restricted to name(args) could one day route to the compiled handler directly, and the baseline tells you what that buys. But the generality isn’t waste on this wire — it’s the protocol’s point: the caller authors the computation, the owner executes it under its own custody, once, atomically.

The mailbox has a resident tier — and it carries a debt

The directory mailbox is the durable tier: cross-process, at-least-once, survives a dead owner. Inside one process the same routed turn has a cheaper transport — std/channel, the bounded MPMC channel that already knows how to carry obligations. The mail record rides the ring as Mail<posted!>, a must-settle debt, and the compiler enforces at build time what the directory can only promise at runtime:

std/proto(Mail) {
    body: string
}

pub tor post { b: string } -> Mail<posted!>
post -> { body: b }

pub tor settle { m: Mail<!posted> } -> i64
settle -> 0

std/channel:new(mail, capacity: 16) { turn: Mail }

std/channel(mail)
! turn m |> drain-turn(m) |> settle(m)

drain-turn borrows the posted mail (Mail<posted> — read without custody) and runs the same work the directory drain runs — cell:run(c, source: m.body) plus the capture commit; the obligation doesn’t cross nested-arm boundaries, so the borrow/settle split is the honest spelling. post mints the debt; send’s | ok consumes the caller’s custody into the channel while | full and | closed retain it — backpressure is a real arm, which a directory cannot express; and the ! turn consumer mints Mail<posted!> fresh at the drain. A consumer that touches a mail without settling it is not a buggy program — it is a refused program:

error[KORU030]: Resource 'm' obligation <posted!> was not discharged. Call one of: deliver, send

Read the refusal’s vocabulary: deliver or send — the debt can forward onward, so an owner that stops owning can hand the mail to the next owner and custody transits with it. celld serializes a Durable Object’s turns by runtime convention; here the drain’s obligation to finish each mail is a compile-time fact. Dropped mail is unrepresentable.

One honest boundary: the mint is seeded by observed custodial sends — a channel fed only by host threads writing the ring raw mints nothing, because the ledger lives at the Koru send, not the vessel. Foreign producers carry bytes; that is the same edge every wire protocol has.

Handles are tokens, not bindings

A *T value crossing into the interpreter — the cell pointer run dispatches on — travels as an opaque token minted by a handle pool: h:Res:1 on the wire, a real pointer only inside the host. Dispatch registers the returned resource, input fields resolve the token back, a discharged handle’s token refuses as spent, and a made-up token never reaches a pointer at all.

What the shape costs

tests/benchmarks/013_cell_node runs the Orleans counter-grain workload — per turn, a run dispatch plus one transaction commit — and tests/benchmarks/015_fleet_mail prices the routed turn the same way: send and deliver timed separately, because in production they are different machines’ clocks. Laptop numbers, 60 turns/mails, checksum-gated (hits == 60 and delivered == 60 read back through the node, or an arm silently never ran):

The decomposition is the honest part: durable minus resident is ~50µs — essentially the whole file floor — and resident itself (~79µs) is the interpreter’s price, since the baseline for a turn with no materializer is noise. The routed turn, 015’s mailbox path: send ~97µs/post against an ~87µs file-lifecycle floor, deliver ~185µs/mail against a ~48µs drain floor — the remainder is the durable turn already priced above plus ~16µs of mail parse and dispatch. Routed end-to-end ≈ ~280µs on the directory tier, ~211µs on the channel tier.

And the pipelined number — the one the gap list used to owe: 015’s pipelined arm runs 8 producer threads claim-posting turn programs into a capacity-16 ring while the pump drains them through the custody-minted consumer above. ~12.9ms for 61 turns — ~4,700 settled turns/second — against celld’s ~789 req/s at 8 connections. Producers genuinely spin on | full, so backpressure is exercised, and every turn still pays the durable commit — transport plus durable turn, not transport alone.

The comparator is Deno celld 0.6.2 running its own examples/counter — celld dev in docker, wrk over HTTP: the durable turn lands ~4.6ms p50 sequential, ~789 req/s pipelined at 8 connections, against a ~172µs wire+V8 floor measured with a no-DO echo path. Sequential turns are exactly-once on both sides; under concurrency celld shows ~1 duplicate turn per connection — the at-least-once its docs admit on ambiguous paths, which is also what the mailbox’s delete-after-return gives you here.

Two named asymmetries, stated rather than smoothed: celld pays an HTTP front koru’s filesystem mail doesn’t, and its acknowledged writes are fsync-grade SQLite commits where durable here is a buffered write (durable+sync spells the same level — on this Mac, APFS fsync costs ~3µs per commit, so the premium is not the sync; celld’s 4.6ms is mostly its own DO/SQLite/replication machinery plus virtio-fs). And the flip side, equally true: celld’s durable write is a real store with fencing built in; state.kv is a flat file wearing the seam a real materializer will own.

So the claim is not “koru is 16-35x faster than celld” — the runtimes don’t share a wire. The claim is narrower and more interesting: the interpreter is not the bottleneck. Koru’s interpreter+custody+materializer tax is ~83µs over its file floor; celld’s machinery tax is ~4,400µs over its own wire floor. The wire-carries-programs design — the thing you’d expect to be expensive — costs less than a tenth of what celld spends just answering HTTP. And under contention the same shape holds: ~4,700 settled turns/s against ~789 req/s, where the koru number is drain-rate-limited — the transport is a ring hop — and celld’s is HTTP+V8+SQLite all the way down.

What it is not, yet

The mailbox is a directory and a ring, not a socket — std/fleet speaks filesystem and std/channel speaks shared memory, so “peer routing” today means nodes on one machine sharing one world or one process; a network wire is the layer the protocol is shaped for but doesn’t have. The channel tier’s debt is in-memory only — a crash mid-ring loses posted mail where the directory tier redelivers; the two-tier spill (| full |> post to the directory) is the spelling the design wants and hasn’t grown yet. capacity evicts an arbitrary resident on activation overflow, not an LRU one. And state.kv remains a flat file: the commit seam is the right boundary, but SQLite or an object store behind it is the thing owed — the same seam where a conditional-write fence will live when ownership checks move from guard-then-write into the commit itself.

The shape survived contact, though: named identity, lexical transactions, custody the compiler enforces, epochs fencing stale owners, and a mailbox where the payload is a program. The interpreter isn’t the cost of this design — it’s what the design is for.