The Compiler That Compiles Itself: Self-Referential Pipeline Overrides

· 8 min read

What if you could write code that instruments its own compilation? Not a compiler plugin. Not a build tool. Just… your code, describing how to compile itself.

Today I want to show you something that, as far as I know, no other compiled language can do.

The Setup

Here’s a complete Koru program:

~import "$std/compiler"
~import "$std/io"

// Override the frontend pass to dump AST
~std.compiler:frontend = std.compiler:frontend.default(ctx)
| ctx c |> std.compiler:ast_dump(ctx: c, flag: "post-frontend") |> ctx c

~std.io:print.ln("Hello from self-inspecting code!")

When you compile this, the output includes:

=== AST DUMP: post-frontend ===
{
  "items": [
    ...
    {
      "type": "flow",
      "invocation": {
        "path": "std.io:print.ln",
        "args": [{ "value": ""Hello from self-inspecting code!"" }]
      }
    }
    ...
  ]
}
Hello from self-inspecting code!

Read that again. The AST dump shows the program that requested the AST dump. The string "Hello from self-inspecting code!" appears in the dumped AST because that’s the program being compiled.

The snake is watching itself eat its tail. And it compiles to a native binary.

How Is This Possible?

Koru’s compiler pipeline is built from abstract events:

// In $std/compiler
~[comptime|abstract] pub event frontend { ctx: CompilerContext }
| ctx CompilerContext

~[comptime|abstract] pub event analysis { ctx: CompilerContext }
| ctx CompilerContext
| failed { ctx: CompilerContext, message: []const u8 }

~[comptime|abstract] pub event optimize { ctx: CompilerContext }
| ctx CompilerContext

Each compiler pass is an event. Events can be overridden. So you can override… the compiler.

But here’s the mind-bending part: the overrides are processed by the very pipeline they’re overriding.

  1. The compiler parses your file
  2. It sees ~std.compiler:frontend = ...
  3. It uses that override to run the frontend pass
  4. Which compiles the file containing that override
  5. Including the override itself

The program is simultaneously the input, the configuration, and the instrumentation of its own compilation.

Composable Instrumentation

Because overrides are independent, you can layer them:

~import "$std/compiler"
~import "$std/time"
~import "$std/io"

// Time the whole pipeline
~std.compiler:coordinate = std.time:now()
| t start |> std.compiler:coordinate.default(program_ast, allocator)
    | coordinated c |> std.time:elapsed(start: start.ns, label: "TOTAL")
        |> coordinated { c.ast, c.code, c.metrics }
    | error e |> error { message: e.message }

// Time just the frontend
~std.compiler:frontend = std.time:now()
| t start |> std.compiler:frontend.default(ctx)
    | ctx c |> std.time:elapsed(start: start.ns, label: "frontend") |> ctx c

// Dump AST after analysis
~std.compiler:analysis = std.compiler:analysis.default(ctx)
| ctx c |> std.compiler:ast_dump(ctx: c, flag: "post-analysis") |> ctx c
| failed f |> failed { ctx: f.ctx, message: f.message }

~std.io:print.ln("Compiled with full instrumentation!")

Output:

⏱️  frontend: 3.24ms
=== AST DUMP: post-analysis ===
{ ... }
⏱️  TOTAL: 18.52ms
Compiled with full instrumentation!

Three independent concerns - timing the whole pipeline, timing one pass, dumping AST - all composing without conflict. Each override fires at its point in the pipeline, does its work, and continues.

But You Can Do More Than Observe

Here’s where it gets truly wild. You’re not limited to observing. You have full AST access. You can transform it.

~std.compiler:optimize = std.compiler:optimize.default(ctx)
| ctx c |> my_custom_optimization(ctx: c)
    | ctx optimized |> ctx optimized

You could write a my_custom_optimization event that:

  • Analyzes hot paths from your production profile data
  • Inlines specific call patterns you know matter
  • Reorders operations for your cache layout
  • Applies domain-specific transforms no general compiler would do

Custom profile-guided optimization. In user space. For a compiled language.

Game engines could add ECS-specific optimizations. Web frameworks could flatten route trees. Scientific code could tune numerical precision. All without forking the compiler.

What Makes This Different

Let me be precise about what’s novel here:

Lisp has macros, but they transform code before the compiler runs. You can’t intercept Lisp’s own compilation passes.

Rust has procedural macros, but they operate on token streams, not semantic AST. And you can’t override rustc’s internal passes.

Zig has comptime, but it’s compile-time evaluation of Zig code. You can’t intercept the Zig compiler itself.

GHC has plugins, but they’re a separate system - Haskell code loaded specially. Not your program naturally configuring its own compilation.

Koru is different:

  1. Same language - The override is Koru code, not a plugin API
  2. Same file - The override lives in the program it affects
  3. Self-referential - The override is compiled by the pipeline it overrides
  4. Composable - Multiple overrides don’t conflict
  5. Full access - You can read AND write the AST
  6. Native output - It all compiles to a binary via Zig

The Implementation Is Tiny

Adding AST dumping to any pass takes two lines:

~std.compiler:frontend = std.compiler:frontend.default(ctx)
| ctx c |> std.compiler:ast_dump(ctx: c, flag: "frontend") |> ctx c

That’s it. No build flags. No plugin registration. No special configuration. Just… code.

Want timing? Same pattern, using $std/time:

~std.compiler:frontend = std.time:now()
| t start |> std.compiler:frontend.default(ctx)
    | ctx c |> std.time:elapsed(start: start.ns, label: "frontend") |> ctx c

The infrastructure is so minimal that instrumentation becomes trivial.

The Strange Loop

There’s something philosophically interesting happening here. The program contains:

  1. Instructions for what to compute (print.ln("Hello"))
  2. Instructions for how to compile those instructions (~std.compiler:frontend = ...)
  3. The ability to observe those instructions being compiled (AST dump)
  4. Including the instructions that requested the observation

It’s self-reference all the way down. Hofstadter’s strange loops, but they run on real hardware and produce real binaries.

The program describes itself describing itself being compiled.

And it works.

Try It Yourself

The full test suite is in the Koru repository:

  • 430_008_pipeline_profiling - Time individual passes
  • 430_009_composed_timing - Time multiple passes independently
  • 430_010_ast_dump_stages - Dump AST at multiple pipeline points

Each test compiles and runs. The instrumentation fires. The self-reference resolves. Native code comes out.

We’re not aware of any other compiled language that can do this. If you know of one, we’d genuinely love to hear about it.

The abstract compiler pipeline and self-referential overrides are available now in the main branch. Import $std/compiler and start introspecting.