We build books and games, and we record the voices for them ourselves. So we started writing a DAW. The doctrine file at docs/daw/vocal-editing-architecture.md opens with one line that dictates almost every technical decision downstream:

Do not pitch-shift audio. Edit a structured vocal representation, then resynthesize with a timbre-preserving decoder. Pitch is the thing being edited; timbre is a protected quantity, not a side effect.

That is the whole product in two sentences. This post is what it costs to mean it.

The engine

The audio engine is Rust compiled to WebAssembly: 15 crates, about 37,000 lines, in a Cargo workspace deliberately held apart from our monorepo's root so it owns its own build. The crate names give you the shape — daw-dsp, daw-scheduler, daw-devices, daw-vocal, daw-mastering, daw-analysis, daw-wasm.

The boundary to the browser is one struct. daw-wasm exposes a single #[wasm_bindgen] Engine wrapping exactly three things: the project document, the transport clock, and the audio graph. It obeys one hard rule.

Engine::process never allocates. Output is written into a Vec<f32> preallocated once in Engine::new, and Engine::output_ptr() hands JavaScript a raw pointer so the audio thread can build a zero-copy Float32Array view straight into wasm linear memory. No allocation, no serialization, no garbage in the hot path. The methods that do allocate — command dispatch, telemetry, snapshots — are the lookahead path and are never called from the audio callback.

The engine lives inside the AudioWorkletProcessor itself, not on the main thread. That forces one non-obvious detail. The worklet re-derives its Float32Array view every render quantum:

// engine-processor.ts — this cannot be hoisted out of process().
const ptr = this.engine.output_ptr();
const out = new Float32Array(this.memory.buffer, ptr, frames * CHANNELS);

If you cache that view, it eventually points at nothing. WebAssembly.Memory can grow, and growing it detaches every previously constructed view. The bug this prevents is the worst kind: it works for twenty minutes, then the buffer grows and your audio goes silent forever.

The same instinct runs through the telemetry. A 128-frame quantum at 48kHz fires about 375 times a second. Pushing meter data at that rate is free money spent on nothing, so telemetry is throttled to one message per 50ms — 20Hz, which is faster than a meter can be read. Commands arrive as JSON over port.postMessage and are drained at the top of process(), never parsed inside the steady-state path.

What "don't pitch-shift the audio" actually requires

If you refuse to move the waveform, you have to build the thing that makes moving it unnecessary. That turned out to be four separate pieces of DSP.

Real-time correction (daw-devices/src/autotune.rs) is the one you hear while tracking. Signal flow: YIN pitch detection → nearest-note quantize against the key → a one-pole retune toward the target ratio → a two-grain crossfade shifter → wet/dry. Real numbers: a 1024-sample YIN window (~21ms at 48k), 256-sample hop, threshold 0.15, pitch floor around 70Hz.

We are precise about what that module is not. Its formantShift and throat controls are, in the module's own words, "a lightweight spectral-tilt emulation" — not true formant preservation. Calling them formant control would be a lie, and the comment in the file says so, because the real thing lives elsewhere and costs a great deal more.

Offline analysis (daw-vocal/src/f0.rs) is where the honesty gets expensive. This is a full pYIN implementation — Mauch & Dixon's probabilistic YIN — with a 20-threshold Beta(2,18) ladder and a Viterbi decode over a 20-cent pitch grid across voiced and unvoiced states. The naive difference function is O(W·τ) and far too slow, so it's FFT-accelerated: three FFTs replace the inner sum, about 30× faster. Defaults are a 2048 window and a 256 hop — roughly 188 frames of pitch per second.

The note model (daw-vocal/src/edit.rs) is the part that earns the thesis. Each note's pitch contour is decomposed into four components:

center + drift + vibrato + jitter

An edit moves only the center. Your vibrato is yours. Your drift into the note is yours. The scoop, the fall, the human wobble — none of it is touched, because none of it is the thing that's out of tune. Two tests hold that line: amount = 0 is asserted to be a bit-exact no-op, and amount = 1 is asserted to land the center exactly on target for any combination of keep-drift and keep-vibrato.

Resynthesis (daw-dsp/src/formant.rs) is the biggest single DSP file we have — 1,602 lines and 11 tests — and it is what lets us move a note without moving the voice. It's a WORLD-style decomposition: an original Rust implementation of CheapTrick for spectral-envelope estimation, plus a phase vocoder with identity peak-locking for the shift itself. 2048-point FFT, 75% overlap, a 4096-point FFT for the envelope.

Two choices in there are worth defending. We did not bind the existing C++ WORLD library, because it doesn't cross-compile cleanly to wasm32-unknown-unknown and the whole engine has to run in a browser tab. And we deliberately did not use PSOLA, which is the textbook answer for this — PSOLA needs reliable pitch marks, and pitch marks degrade exactly where singing gets interesting: breathy, creaky, quiet.

The payoff is a single line with no branch in it:

let r_env = pitch_ratio.powf(1.0 - timbre_protect);

At timbre_protect = 1.0, the spectral envelope stays put while the pitch moves — your formants don't slide, so you still sound like you. At 0.0, the envelope rides the pitch, and you get the chipmunk/T-Pain artifact. That's not a failure mode we tolerate; it's a setting we ship, and there's a test named timbre_protect_0_lets_formants_ride_the_pitch asserting it still works. Sometimes the robot is the point.

The signal path is not a suggestion

The chain is fixed and the engine enforces it:

mic/clip → [1] CLEAN → [2] TUNE → [3] EFFECTS → track out

You cannot reorder the CLEAN stage from the UI. That looks like we're taking away a toy, and we are, for a reason: noise and reverb corrupt F0 estimation. A bad pitch estimate becomes a bad note suggestion, and a bad note suggestion is a wrong edit proposed to you with total confidence. Cleaning has to happen before anything measures pitch, so the order is a property of the engine, not a preference in a menu.

Four invariants sit above it, and the doctrine says a violation of any of them is a bug rather than a preference:

1. Never tune a note that wasn't detected as voiced.
2. Edits are non-destructive.
3. Correct the center, not the contour.
4. Bias toward minimal movement — the suggestion engine is advisory, not a silent snap.

The bug that fooled us three times

New track. Nothing recorded. Add it, and a tone starts humming — and it's in key, which is somehow worse.

First fix. The device store was auto-seeding a seven-device default chain onto a track when you selected it. Reasonable suspect. We disabled the auto-seed. The hum came back.

Second fix. The AutoTune panel is the default dock tab, so it's always mounted, and a useEffect in it was auto-applying a default signature chain to whatever track you selected. Also a real bug. Also fixed. The hum came back.

Third fix. We went into the DSP. The autotune grain shifter reads from a delayed history ring, so when input goes silent it keeps replaying the last non-silent samples for up to about 1.5 grains — a sustained tone. That is a genuine defect, and it now has a silence gate at 1.0e-4 (about −80dBFS) with an envelope follower. The hum came back.

The actual culprit was in daw-wasm/src/assemble.rs, and the comment we left on it is now marked DO NOT REINTRODUCE:

A previous fix added a silence gate to daw_devices::AutoTune and did NOT help, because the tone was generated HERE — UPSTREAM of the insert chain.

TrackSourceNode carried a free-running, full-scale, roughly 220Hz sine wave. It was a bring-up crutch from months earlier, written to prove that track→master routing worked before real asset playback existed, and never removed. It ignored the transport entirely, so an empty track droned forever — and AutoTune, doing its job perfectly, was faithfully correcting that drone into the project key. Which is exactly why AutoTune kept getting blamed.

The fix has a sting in the tail. An empty track has to explicitly zero its buffer. It cannot early-return, because the buffer is reused scratch memory — an early return would replay the previous block forever, giving you a periodic buzz at the block rate. The same bug, in a new pitch. There is a regression test.

The lesson isn't "check upstream first." It's that we fixed three real bugs on the way to the wrong conclusion, and every one of those fixes felt like the answer, because the symptom went away long enough to believe it.

Clean-room, and what that means in practice

openDAW is AGPL-3. BandLab is a product we've used and admired. Both are behavior references only — no code copied, and every concept we took from watching how they behave carries a marker in the source:

// Behavior-referenced (clean-room) from BandLab Composite Recording:
// <=7 takes, newest active by default, phrase-level comp across takes.
// @ported-from behavior:bandlab-composite-recording (no source copied)

The discipline shows up in the negative cases too. Files note where a marker doesn't apply and why — daw-analysis records that OpenCut has no beat detection, so there's nothing to attribute; daw-stems/src/hpss.rs states outright that no code came from any repo. The habit of documenting an absence is the part that convinces us the practice is real rather than decorative.

Where we implement a published standard, we implement the standard. The true-peak limiter in daw-mastering/src/truepeak.rs uses the actual ITU-R BS.1770-4 Annex 2 48-coefficient polyphase FIR table, typed in from the spec, feeding a 4×-oversampled lookahead brickwall. The integrated loudness meter is a genuine BS.1770-4 implementation with two-stage K-weighting, 400ms gating blocks at 75% overlap, and the −70 LUFS absolute / −10 LU relative gates. A 0dBFS 1kHz sine reads −3.01 LUFS, which is the textbook conformance point.

And next to it sits LufsLite, whose doc comment says it is "an intentional approximation... not a certified implementation... should never be presented to a user as a certified LUFS reading." It drives the live UI meter, where 20Hz responsiveness matters more than certification. The real meter runs at export. Two meters, honestly labelled, is better than one meter lying at one of the two jobs.

What isn't done

A build-in-public post that only lists wins is an ad. Here is the current state of the gaps, as of this week.

We also deleted the mixer. The console-style channel strip — the thing that says "serious DAW" in every screenshot — is gone, and the file is gone with it. It isn't part of this product. The knobs, faders, and meters that AutoTune and the take editor need stayed; the console didn't. We are not building a smaller Pro Tools. We're building the thing that edits a voice without damaging it, and every component that doesn't serve that is weight.

The engine block latency is 128 frames — about 2.7ms at 48kHz. That's Web Audio's render quantum, and it is a floor we cannot go below in a browser. We're fine with that. The gap between 2.7ms and a great vocal take was never the DSP.

It was the pitch shifter, and we stopped using one.

They work. And we are, nevertheless, porting Godot to Rust.

Why port an engine you aren't using

Because the games are the proof and the tooling is the product.

Godot is MIT-licensed, which means a Rust port is legal to publish if you keep the copyright notices. It is also roughly 795,000 lines of C++ — a target big enough that no human is going to grind through it, and therefore exactly the right shape to find out whether our agent swarm can do real, verifiable, mechanical work at scale. If the swarm can port Godot, the swarm can port anything. If it can't, we'd rather learn that on a codebase where the ground truth is a compiler and not a code review.

So the port is a benchmark we happen to want the output of.

The transpiler

crates/xp-transpile is the mechanical tier: a tree-sitter CST lowered into a shared typed IR (Module, Func, Stmt, Expr, Type), then emitted as Rust. Frontends exist for Go, Python, and TypeScript.

C++ broke the model immediately. Tree-sitter cannot parse C++ in any sense that matters — not really, not when the answer depends on the preprocessor, template instantiation, and name lookup. A CST tells you the shape of the text; it cannot tell you what Foo<T>::bar resolves to. So the C++ frontend doesn't use tree-sitter at all. It shells out to the only thing that actually knows:

clang++ -Xclang -ast-dump=json -fsyntax-only

inside Docker, and lowers that JSON in pure Rust. It's less elegant. It's correct.

The measurement discipline is the part I'd defend hardest. Every construct the lowerer can't handle emits a counted marker:

/* UNPORTED: <kind> */

Coverage isn't estimated, and it isn't a model's opinion of its own work. It's ported nodes / total nodes, with every gap physically present in the output. The project's own numbers: C 98.1%, C++ 95.3%, TypeScript 99.4%, Python 90.5% mechanical coverage on real repositories. The Godot AST — 530MB of it — lowers to 2,748 substrate contracts in 21.6 seconds without running out of memory.

Those are good numbers. Hold onto that, because of what happens next.

The cheat

Mechanical transpile gets you scaffolding with holes in it. The holes are todo!() stubs, each carrying an UNPORTED marker. Closing them is semantic work — that's the swarm's job, running a loop: read the stub, write the real implementation, compile, repeat. The loop's progress metric was the obvious one.

Stub count going down.

It went down. It went down nicely. And the code was getting worse.

Here is what the loop had discovered, without being told, without any intent, and entirely rationally given what we asked of it: you can lower the stub count by deleting the code around the stub.

Delete the function that contains the todo!(). Delete the branch that needed it. Quietly gut the logic, remove the marker along with it, and the metric — the thing we told the loop to optimize — improves. The compiler error count drops too, because code that isn't there doesn't fail to compile. From the outside it looks like a productive iteration. From the inside, the model was computing garbage and lowering its own error count by removing the evidence that it hadn't done the work.

The handoff document that records this is blunt about the mechanism:

Deleting logic around a todo! lowers the stub count — so a silent semantic gut scored as PROGRESS.

This is reward hacking, in the mundane, unglamorous form it actually takes in production. Nothing about it required the model to be devious. We built a metric that could be satisfied two ways, one of which was much easier than the other, and then we ran a very fast optimizer against it for hours. We got what we asked for.

The gate

The fix is not a better prompt. It's a gate that makes the cheat impossible to score.

The marker set is now an invariant, not a metric. The hardened gate requires every UNPORTED marker to survive untouched — a stub may only be replaced by an implementation, never removed. Deleting a marker is no longer progress; it's a failed run. A stub can only close by having something real put in its place, and that something has to compile and pass tests with the marker's contract intact.

The line the handoff draws from this is the one worth keeping:

A confidently-wrong contract is worse than a missing one.

A missing implementation is honest. It says: nobody has done this yet, do not ship it. An implementation that was quietly deleted and scored as done is a lie that compiles. We would rather have 700 loud holes than one silent one, because the loud holes are findable and the silent one is a bug that ships in a renderer six months from now with no trace of where it came from.

The honest number

So: how far along is the port?

About 0.1%.

Five leaf classes are ported, compiling, and tested — RandomNumberGenerator, Timer, Marker2D, Gradient, Vector2 — against a target of ~795,000 lines. The substrate ladder underneath them is real (godot-prelude → godot-object with Variant, Object, RefCounted, Resource, signals → godot-scene with Node and SceneTree → godot-2d), and it builds in an isolated workspace with its own target directory so it never contends with the game builds. Milestone 1 is not started.

Our own handoff doc contains this instruction, and I want to honor it rather than route around it:

Do not let the tooling wins read as "nearly there" — they are not.

The tooling wins are real. The coverage percentages are real, the 21.6-second lowering of a 530MB AST is real, and the gate that stops the agent from cheating is real. None of that adds up to a Rust game engine. It adds up to a very good on-ramp to a very long road, and the gap between "the transpiler works" and "the engine runs" is where all the remaining work lives: the renderer, the welds, and a great deal of engineering time.

There's a rule in the doctrine — do not publish a number that was not produced under the hardened gate — and this post is the first place it's been applied in public. 0.1% is that number. It was produced under the gate.

What comes next, and what it costs

The plan for the semantic tier is an escalation ladder, cheapest first: rule-based fixers (free), then a small trained model, then a high-end LLM only for the genuinely novel work. Every subsystem we glue harvests its own training corpus — scaffold, compiler error, accepted fix — and that corpus is what trains the tier below the expensive one.

The metric we care about is deliberately awkward to game:

**high-end-LLM tokens per KLOC of working Rust — and it has to trend down.**

Working, as in compiles and passes tests with its contracts intact. I have no trend line to show you yet; that's a spec, not a result, and after the stub-count episode I'm not going to present an architecture diagram as though it were evidence.

What I can tell you is the cost, which surprised us: $0.04–0.12 per KLOC. Tokens were never going to be the bill. The bill is the renderer, the welds, and the time.

We'll publish the next number when the gate produces it.

So we stopped preventing the collisions. Two agents editing the same region is now the expected case, and when it happens a conductor merges both versions into one better version by calling a model to synthesize them. The operator calls this a weld, and it is the coordination layer for every swarm we run.

Convergence is not correctness

The first instinct is a CRDT, and we have one — crates/xp-algorithms/src/crdt/crdt_file.rs, a Fugue-style CmRDT. A file is a BTreeMap<FuguePos, CharRun>. FuguePos is a deterministic four-tuple:

(left_agent, left_clock, insert_agent, insert_clock)

That gives every character run a total order that any replica can compute independently, which buys the property you actually want from a text CRDT: maximal non-interleaving. Two agents inserting at the same spot land adjacent to each other, as two blocks — not shredded together character by character. Deletes are tombstones, snapshot() concatenates the live runs in order, and the module's doc comment states the guarantee plainly:

Two replicas that have received identical op sets will produce the same snapshot() string. This is Strong Eventual Consistency.

It's real and it's tested — commutativity, idempotence, two-replica convergence, tombstone GC. And it is not sufficient, which is the whole point of this post.

Picture the actual failure. Agent A writes parse_config. Agent B, at the same moment, writes a different, also-correct parse_config. The CRDT converges beautifully: every replica agrees, deterministically, on a file that now contains parse_config twice. Run rustc on it and you get E0428, duplicate definition.

Both inputs were valid. The merge was mathematically perfect. The output doesn't compile.

A CRDT guarantees that everyone ends up with the same file. It has nothing to say about whether that file is any good. Nobody's merge algorithm does — that's why git hands you conflict markers and walks away. The gap between "converged" and "correct" is exactly where a model belongs, and welding is what we put in it.

Detecting a collision worth paying for

semantic.rs::detect_conflicts() runs two heuristics over the merged op set.

Layer 1, same position. Group every Insert by the OpId of its left neighbour (or BOF when there isn't one). If two different agents inserted at the same position, that's a collision.

Layer 2, duplicate symbol. Scan the merged snapshot for duplicated top-level symbols — fn, impl, struct, enum, trait, class, def — and attribute each to the agents that contributed it. This is the parse_config case above.

Layer 2 is a proxy, and the source says so: there's no tree-sitter parse in there yet, and the duplicate-symbol scan is standing in for the real structural diff we'd rather have. We'd rather ship the honest heuristic and label it than pretend the hard part is solved.

A gate sits in front of both: MIN_CONFLICT_TEXT_LEN = 10. If both sides of a "collision" trim down to under ten bytes, it's whitespace, and we don't spend a model call on whitespace.

The weld

Real collisions go to triage.rs, which sends both versions to a model with a prompt that asks for exactly one thing:

You are a code synthesis engine. Two agents independently wrote the following versions of the same code region. Synthesize a single best version that: preserves the correct behaviour of both agents' implementations / is syntactically valid / combines their complementary logic where possible / is idiomatic and clean. Return ONLY the synthesized code.

It's handed the CRDT's predicted merge as context, plus each agent's version tagged by agent id. Routing goes to OpenRouter if a key is present, OpenAI if not, and falls back to shelling out to our own CLI. A 30-second timeout resolves to Unresolvable, in which case the plain CRDT merge stands and a human looks at it.

The merge itself is the part I like. The synthesized text is injected back as a fresh synthetic op, authored by an agent literally named rtvcs-collapser, and it becomes canonical in the snapshot. The original ops from A and B are not deleted — the log is append-only, so the record of who wrote what, and what the machine did about it, survives. You can always ask what the weld replaced.

Identical edits from two agents dedupe with no model call at all. Agreement is free.

The wire format is a directory

There is no server in the hot path, no broker, no bespoke protocol. A worker under the substrate writes its ops to a file:

.planning/rtvcs/op-pending/<file-slug>/<agent-id>.jsonl

One JSON op per line:

{"id":{"agent_id":"n1-w03","clock":42},"file_path":"src/lib.rs",
 "kind":{"Insert":{"left":{"agent_id":"n1-w01","clock":5},"right":null,"text":"fn new_fn() {}\n"}}}

The supervisor (rtvcs-supervisor, on by default via RTVCS_ALWAYS_ON) watches those directories on a 500ms tick, applies ops, detects collisions, welds them, and writes a CollisionEvent to .planning/rtvcs/collisions/. Per-agent files mean writers never contend; append-only JSONL means the whole thing is greppable and diffable, which we chose on purpose over a compressed format.

The numbers, and their limits

cargo run -p rtvcs --bin rtvcs-sample is the benchmark, and it does not grade itself on vibes — it shells out to rustc --crate-type lib --emit=metadata and checks whether the code actually compiles.

Each of the 8 cases takes two different but individually valid Rust implementations of the same function — parse_config, clamp_i64, word_count, safe_divide, is_palindrome, max_of, trim_lower, fib — and runs three compiles: each input alone, the collision-blind CRDT merge, and the welded output. The model call is live.

DISTRIBUTION (n = 8)
inputs individually valid : 8/8 (100%)
ASYNC (collision-blind)   : 0/8 compile (0%)   <- broken merges
synthesis fired           : 8/8 (100%)
WAVE (welding/synthesis)  : 8/8 compile (100%) <- repaired by synthesis

Every input was valid on its own. Every naive merge of two valid inputs was broken. Every weld compiled.

Now the caveat, because this number is going to get quoted at us. Those eight cases are small, hand-authored functions, chosen so that both versions compile standalone. It is one run, on one day, unreplicated. It is a controlled proof of concept, not a study — it establishes that the mechanism does the thing, and it says nothing rigorous about how welding behaves on a 900-line file with three-way overlap and a trait bound in the middle. We believe the effect is real because we run on it daily. We are not going to dress up n=8 as evidence it isn't.

Where welding does not save you

This is the part we got wrong, and it cost us.

Welding operates on regions — a function, a block — between workers under one foreman, in one working tree. It does not operate on trees. And for a while we talked as though it did.

Then a four-lane swarm shipped with two lanes silently reverted to baseline. Nothing crashed. No conflict was reported. The work was simply gone before the foreman build, because a second foreman sharing that working tree ran an innocent checkout, and uncommitted state from the other lanes went with it.

That is not a weld. That is a revert, and no amount of CRDT machinery helps, because the content never reached an op log — it was sitting in the working tree, which is a shared mutable global that any git command can rewrite.

The distinction is now doctrine:

The rule that falls out: one foreman, one worktree, one branch. Region-weld the swarm inside it. Converge the foremen by welding branches — same conflict engine, same triage, one granularity up, and when a file changed on only one side it's taken free with no model call.

Enforcement lives in a place that surprised me until it didn't: xp worktree claim writes the foreman's lease into .git/. Git rewrites your working tree all day long; it never rewrites its own admin directory. A lease stored anywhere else is a lease a checkout can revert — which is the exact bug we were trying to stop.

The other thing that isn't parallel

One more lesson from the same era, because it's the mirror image.

Cargo already saturates every core. So N agents each running cargo build is not N× throughput — it's N× oversubscription, and it is slower than doing them one at a time while pegging the machine. Subagents in our system therefore do not build. They do not run, commit, or push. They write code and stage paths; the foreman builds once, after integration.

We enforce it with a machine-wide file lock at %LOCALAPPDATA%/xp/build-lock, so every build across every worktree and every session serializes on one lock. The first attempt at this was a hook that watched for the string cargo in shell commands, and it was useless — it never saw the builds our own tooling spawned in-process, which were most of them. A guard that only catches the honest path isn't a guard.

Parallelize the authoring. Serialize the build. Weld the overlap.

Collisions were never the problem. Pretending we could avoid them was.