Opaque Taps: How to Profile a Profiler (Without the Universe Exploding)

· 10 min read

The Observer Problem

Here’s the situation: We have a profiler that observes every event transition in your program. One import, full program profiling:

~[profile]import "$std/profiler"

The profiler uses a universal tap pattern to observe everything:

// Observe ALL transitions
~* -> *
| Profile p |> write_event(source: p.source, timestamp_ns: p.timestamp_ns)
    | done |> _

That ~* -> * means “observe ALL transitions.” Beautiful. Powerful. Complete.

But wait. The profiler itself is implemented as Koru events. Writing profile data is an event. Which means…

The profiler observes itself.

Which triggers more profiling events. Which trigger more profiling events. Which trigger more profiling events.

Infinite recursion. Stack overflow. Universe explodes.

The Naïve Solutions (All Bad)

Option 1: Hardcode an Exception

~* -> * | Profile p where p.source != "profiler" |> write_event(...)

NO. This is disgusting. The profiler shouldn’t need to know its own implementation details. What if we rename something? What if we refactor? This is fragile, hardcoded garbage.

Plus, it violates the philosophy: No special cases. No magic.

Option 2: Disable Taps During Profiling

~* -> * | Profile p |> disable_taps() |> write_event(...) |> enable_taps()

NO. This requires global state mutation. It’s not composable. What if you have two profilers? What if another tap needs to run? This is a layering violation and a concurrency nightmare.

Option 3: Accept the Recursion

// Just let it recurse, but only profile the top-level call
~* -> * | Profile p where !p.is_nested |> write_event(...)

NO. This loses information. You’d never see the actual cost of profiling itself. Plus, tracking “is_nested” requires runtime state. We’re back to mutable globals.

All of these solutions suck. They’re Band-Aids on a fundamental architectural problem.

The Insight: Opacity is a Property of Observation

The problem isn’t that the profiler observes itself. The problem is that some taps should be invisible to other taps.

Not because they’re special. Not because we hardcoded an exception. But because opacity is a semantic property of the observation itself, not the event.

Think about it: When you write a profiler, you know that the profiling instrumentation itself shouldn’t be profiled. That’s not an implementation detail. That’s a semantic invariant about the observation.

The profiler’s taps should be opaque to other taps.

The Solution: ~[opaque] on Taps

The key insight: Taps can observe other taps.

Koru allows nested event taps - one tap can observe the continuation flow of another tap. This is insanely powerful for meta-instrumentation (like profiling your profiler’s overhead).

But sometimes you want a tap to be invisible to this observation. That’s what ~[opaque] does:

// These taps are opaque - invisible to other taps
~[opaque]tap(koru:start -> *)
| Profile p |> write_header()
    | done |> _

~[opaque]tap(* -> *)
| Profile p |> write_event(source: p.source, timestamp_ns: p.timestamp_ns)
    | done |> _

~[opaque]tap(koru:end -> *)
| Profile p |> write_footer()
    | done |> _

When the compiler processes taps, it filters: “Is this tap opaque?” If yes, other taps don’t observe its transitions.

No runtime overhead. No global state. No special cases.

The filtering happens at compile time. Opaque taps simply don’t match other tap patterns. The generated code never even checks.

Why Mark the TAP, Not the Event?

You might wonder: Why put [opaque] on the tap instead of the event?

Because events are just events. They’re not inherently “meta” or “instrumentation.” The write_event() function is just a regular event that writes JSON to a file.

What makes it special is how it’s observed. The tap is what’s doing the meta-level instrumentation, so the tap is what needs to be opaque.

This enables powerful composition:

// Regular event - just writes to a file
event write_event(source, timestamp) { ... }

// Profiler tap - opaque to other taps
~[opaque]tap(* -> *) | Profile p |> write_event(...)

// But you could also observe write_event directly!
~write_event -> * | Log l |> debug.log("Profile written")

The same event can be used in both opaque instrumentation (the profiler tap) and regular observable code (the debug tap). Opacity is a property of observation, not events.

What This Unlocks

1. Composable Profilers

You can have multiple profilers, each opaque to the others:

~[profile]import "$std/profiler"        // Profiles everything except itself
~[trace]import "$std/execution-trace"   // Traces everything except itself
~[memory]import "$std/memory-profiler"  // Tracks allocations except its own

They coexist peacefully. No conflicts. No recursion.

2. Self-Instrumenting Libraries

Libraries can include their own instrumentation without worrying about observer effects:

// HTTP library with built-in request logging
~[opaque]tap(http.request -> *)
| Request r |> log.request(r)

The logger observes HTTP requests. But other taps don’t observe the logger. Perfect layering.

3. Meta-Profiling When You Need It

Because opacity is optional, you can choose to profile the profiler when you want to measure overhead:

// Profile the profiler's overhead by NOT making it opaque
~[profile|debug]tap(* -> *)
| Profile p |> write_event(...)  // No [opaque]!

// Meta-profiler observes the profiler
~[meta-profile]tap(* -> *)
| Profile p |> measure_overhead(...)

Now the meta-profiler sees the profiler’s transitions and can measure its actual cost. Opacity is under your control.

4. Metacircular Compiler Development

The compiler can instrument itself during compilation:

// Compiler profiling its own passes
~[comptime|profile]import "$std/profiler"

~parse -> shape_check | AST ast |> validate(ast)

The profiler observes the compiler passes. But the profiler’s own events are opaque. The compiler profiles itself without infinite recursion.

This is metacircular development. The tool instruments itself. The insights feed back into optimization. The loop closes.

The Philosophy: Semantic Properties, Not Magic

What makes this solution beautiful isn’t the technical implementation. It’s the philosophy.

Opacity isn’t a hack. It’s not a special case. It’s a semantic property that the author declares about how observation happens.

When you write:

~[opaque]tap(* -> *) | Profile p |> write_event(...)

You’re saying: “This observation is part of the meta-layer. It observes other code, but other taps shouldn’t observe it.”

The compiler respects that declaration. The tap transformer filters it out. The profiler works without recursing.

And because it’s a compile-time property, there’s zero runtime cost. Opaque taps simply don’t generate nested observation code.

The Implementation: A Week of Breakthroughs

This opaque taps feature is part of a remarkable week of progress on the Koru compiler:

48 hours. 30 commits. Multiple architectural breakthroughs.

  • Opaque Taps - Solved profiler recursion with semantic properties
  • Type Registry Integration - The compiler tracks types that the backend can query
  • Compiler Requirements Collection - The compiler declares what it needs and generates build.zig automatically
  • Self-Documenting Compiler Flags - The compiler tells you how to use it with dynamic help discovery
  • The Ruthless Cleanup - Deleted 79,000 lines of archived code and stale artifacts

The pattern is consistent: The compiler is becoming self-describing and self-instrumenting.

It doesn’t just compile your code. It tells you what it knows. It profiles its own performance. It generates its own build system.

And opaque taps are the key that makes this possible without infinite recursion.

The Collaboration: AI-First Development

This breakthrough happened through human-AI collaboration:

  • Human vision: “I want a profiler that profiles everything without recursing.”
  • AI design exploration: “What if opacity is a property of observation?”
  • Shared implementation: 30 commits over 48 hours, alternating insights and code
  • Honest testing: The regression suite kept us honest. No green checkmarks for broken code.

The result is code that neither could have written alone. The human brought domain expertise and vision. The AI brought implementation speed and systematic thinking.

This is what AI-first development looks like. Not AI replacing humans. Humans and AI building together, each doing what they’re best at.

Why Delete 79,000 Lines?

The other major milestone from this week: We deleted 79,000 lines of archived test code and stale artifacts.

Why? Because git remembers. The history is safe. But the current codebase should only contain living code that serves the current architecture.

Every time we pivot, we delete the old approach. No “just in case” code. No “maybe we’ll need this later” cruft.

This is possible because:

  1. Git is our archive - History is never lost
  2. Tests are our confidence - If tests pass, the refactor worked
  3. AI can resurrect - If we need old code, AI can pull it from git history

The result: A codebase that stays lean, honest, and comprehensible.

The Punchline

We set out to build full-program profiling in one line:

~[profile]import "$std/profiler"

But the profiler observed itself, which caused infinite recursion.

We could have added a hardcoded special case. We could have hacked around it.

Instead, we asked: “What semantic property makes observation invisible to other observation?”

The answer: Opacity on the tap itself.

Now the profiler profiles everything except itself. Zero runtime overhead. No special cases. No magic.

And the same mechanism enables:

  • Composable profilers that coexist peacefully
  • Self-instrumenting libraries without observer effects
  • Meta-profiling when you want to measure overhead
  • Metacircular compiler development

One feature. Infinite possibilities.

This is what happens when you design the language for semantic properties, not special cases.

This is what happens when you delete the cruft and keep the codebase honest.

This is what happens when humans and AI collaborate on hard problems.

We built a profiler that profiles itself without infinite recursion.

The universe did not explode.

The future is bright.

Want to try Koru? Check out the language guide or read about full-program profiling in more detail. The compiler is open source, the tests are real, and the philosophy is infectious.

This is an AI-first project. Every feature is designed through human-AI collaboration. If that excites you, join us.