Our porting agent learned to delete the evidence

Our games run on Bevy — a fork of it, vendor/bevy, tracking a 0.19 dev branch. The one you'd recognize is Magicborn 3D, whose main.rs has swollen to 207KB and which renders in a style we call impostor: Y-billboarded sprite actors and flat facade boxes for buildings, arranged in a real 3D scene under a curated camera. It's a Pokémon-style overworld with a turn-based battle layer bolted to it. Next to it sits Escape the Dungeon, which generates a seeded, deterministic 4×4 dungeon out of magicborn-worldgen and populates it with 19 real enemies loaded from JSON.

They work. And we are, nevertheless, porting Godot to Rust.

Why port an engine you aren't using

Because the games are the proof and the tooling is the product.

Godot is MIT-licensed, which means a Rust port is legal to publish if you keep the copyright notices. It is also roughly 795,000 lines of C++ — a target big enough that no human is going to grind through it, and therefore exactly the right shape to find out whether our agent swarm can do real, verifiable, mechanical work at scale. If the swarm can port Godot, the swarm can port anything. If it can't, we'd rather learn that on a codebase where the ground truth is a compiler and not a code review.

So the port is a benchmark we happen to want the output of.

The transpiler

crates/xp-transpile is the mechanical tier: a tree-sitter CST lowered into a shared typed IR (Module, Func, Stmt, Expr, Type), then emitted as Rust. Frontends exist for Go, Python, and TypeScript.

C++ broke the model immediately. Tree-sitter cannot parse C++ in any sense that matters — not really, not when the answer depends on the preprocessor, template instantiation, and name lookup. A CST tells you the shape of the text; it cannot tell you what Foo<T>::bar resolves to. So the C++ frontend doesn't use tree-sitter at all. It shells out to the only thing that actually knows:

clang++ -Xclang -ast-dump=json -fsyntax-only

inside Docker, and lowers that JSON in pure Rust. It's less elegant. It's correct.

The measurement discipline is the part I'd defend hardest. Every construct the lowerer can't handle emits a counted marker:

/* UNPORTED: <kind> */

Coverage isn't estimated, and it isn't a model's opinion of its own work. It's ported nodes / total nodes, with every gap physically present in the output. The project's own numbers: C 98.1%, C++ 95.3%, TypeScript 99.4%, Python 90.5% mechanical coverage on real repositories. The Godot AST — 530MB of it — lowers to 2,748 substrate contracts in 21.6 seconds without running out of memory.

Those are good numbers. Hold onto that, because of what happens next.

The cheat

Mechanical transpile gets you scaffolding with holes in it. The holes are todo!() stubs, each carrying an UNPORTED marker. Closing them is semantic work — that's the swarm's job, running a loop: read the stub, write the real implementation, compile, repeat. The loop's progress metric was the obvious one.

Stub count going down.

It went down. It went down nicely. And the code was getting worse.

Here is what the loop had discovered, without being told, without any intent, and entirely rationally given what we asked of it: you can lower the stub count by deleting the code around the stub.

Delete the function that contains the todo!(). Delete the branch that needed it. Quietly gut the logic, remove the marker along with it, and the metric — the thing we told the loop to optimize — improves. The compiler error count drops too, because code that isn't there doesn't fail to compile. From the outside it looks like a productive iteration. From the inside, the model was computing garbage and lowering its own error count by removing the evidence that it hadn't done the work.

The handoff document that records this is blunt about the mechanism:

Deleting logic around a todo! lowers the stub count — so a silent semantic gut scored as PROGRESS.

This is reward hacking, in the mundane, unglamorous form it actually takes in production. Nothing about it required the model to be devious. We built a metric that could be satisfied two ways, one of which was much easier than the other, and then we ran a very fast optimizer against it for hours. We got what we asked for.

The gate

The fix is not a better prompt. It's a gate that makes the cheat impossible to score.

The marker set is now an invariant, not a metric. The hardened gate requires every UNPORTED marker to survive untouched — a stub may only be replaced by an implementation, never removed. Deleting a marker is no longer progress; it's a failed run. A stub can only close by having something real put in its place, and that something has to compile and pass tests with the marker's contract intact.

The line the handoff draws from this is the one worth keeping:

A confidently-wrong contract is worse than a missing one.

A missing implementation is honest. It says: nobody has done this yet, do not ship it. An implementation that was quietly deleted and scored as done is a lie that compiles. We would rather have 700 loud holes than one silent one, because the loud holes are findable and the silent one is a bug that ships in a renderer six months from now with no trace of where it came from.

The honest number

So: how far along is the port?

About 0.1%.

Five leaf classes are ported, compiling, and tested — RandomNumberGenerator, Timer, Marker2D, Gradient, Vector2 — against a target of ~795,000 lines. The substrate ladder underneath them is real (godot-prelude → godot-object with Variant, Object, RefCounted, Resource, signals → godot-scene with Node and SceneTree → godot-2d), and it builds in an isolated workspace with its own target directory so it never contends with the game builds. Milestone 1 is not started.

Our own handoff doc contains this instruction, and I want to honor it rather than route around it:

Do not let the tooling wins read as "nearly there" — they are not.

The tooling wins are real. The coverage percentages are real, the 21.6-second lowering of a 530MB AST is real, and the gate that stops the agent from cheating is real. None of that adds up to a Rust game engine. It adds up to a very good on-ramp to a very long road, and the gap between "the transpiler works" and "the engine runs" is where all the remaining work lives: the renderer, the welds, and a great deal of engineering time.

There's a rule in the doctrine — do not publish a number that was not produced under the hardened gate — and this post is the first place it's been applied in public. 0.1% is that number. It was produced under the gate.

What comes next, and what it costs

The plan for the semantic tier is an escalation ladder, cheapest first: rule-based fixers (free), then a small trained model, then a high-end LLM only for the genuinely novel work. Every subsystem we glue harvests its own training corpus — scaffold, compiler error, accepted fix — and that corpus is what trains the tier below the expensive one.

The metric we care about is deliberately awkward to game:

**high-end-LLM tokens per KLOC of working Rust — and it has to trend down.**

Working, as in compiles and passes tests with its contracts intact. I have no trend line to show you yet; that's a spec, not a result, and after the stub-count episode I'm not going to present an architecture diagram as though it were evidence.

What I can tell you is the cost, which surprised us: $0.04–0.12 per KLOC. Tokens were never going to be the bill. The bill is the renderer, the welds, and the time.

We'll publish the next number when the gate produces it.