# Lab 011 — Raw lab detail: Buy the labor, not the judgment

Working notes behind the published entry. Rougher than the writeup, kept honest. The full
decision record lives in the repo specs; this is the load-bearing subset.

## The one question

The pitch says a local bug-fix agent needs a frontier tier in the loop to escalate to. Fine:
escalate to it where? The loop has two seats, the worker that writes the patch and the
controller that diagnoses failures. The lab holds one loop, one deterministic test gate, and
one feedback contract constant across 70 real bug-fix pull requests and swaps only the models
in those two seats. Whichever seat the frontier earns, that is where the money goes.

## The instrument

Seventy pull requests mined by git archaeology from real Python libraries (click, requests,
dateutil, more-itertools, prettytable, and others), each shipping the fix's own test suite as
an unfalsifiable pass or fail. The benchmark supplies the touched files, so this measures
localized repair, not repository search. Worker default: Gemma 4 26B (NVFP4) on an owned DGX
Spark. Hosted seats: gpt-5-mini and GPT-5.5, metered. One stateful loop, four turns, rich
deterministic feedback. The gate decides done. No model judges its own work.

## The runs

- Free local worker, first attempt: 48 of 70. The 22 misses become the pool every later lab
  inherits.
- Free local worker, stateful loop on the 22: 14 recovered on deterministic feedback alone.
  Most first-attempt misses were execution failures, not capability gaps.
- Frontier controller diagnosing the local worker: no net task over the free loop. Three
  seeds, matched tasks: 13-13, 10-10, 11-10. Never once ahead. Roughly 98K frontier tokens a
  run spent to draw even.
- Trajectory-aware controller (sees the full attempt history): 13-11 on one 14-task subset.
  A lead inside the sampling band, not a finding. One more seed decides it.
- Frontier model moved into the worker seat, same harness: 20 of 22, clearing three of the
  five hardest residual tasks.
- Escalation ladder inside the harness: a hosted mini worker under frontier control cleared
  one residual task; the frontier worker cleared three. The last two held out.
- A fifth run put a top model inside Claude Code and cleared all 22. Held out of the
  comparison: it swapped the harness and the model at once and proves neither.

## The rulings

The controller seat buys nothing. The worker seat buys everything the tier has to sell. A
weak worker cannot execute a good diagnosis, so buying the frontier's judgment as loop
control is buying the wrong thing. The deterministic evaluator keeps the determines-done
authority in both cases, which is what makes the seat-swap safe to run at all.

## The economics observation that outgrew its footnote

The cost flips with how you buy tokens. Metered, you reserve the frontier for the hard tail.
On a flat subscription its labor is free up to the cap, so you run it as the worker. During
these runs the metered account hit its quota and stopped the experiment mid-run; the
flat-plan arm ran through. The pricing regime is a first-class variable. Lab 012 made it the
headline.

## Operational record (mistakes included)

The residual-five audit matters more than it looks: before trusting any residual, earlier
work found most loop failures in an adjacent pool were broken fixtures, not model failures.
This pool was certified by a reproduce-or-drop gate for exactly that reason. Seed variance
was measured, not assumed: borderline tasks flip between identical runs, which is why every
controller claim above is stated on matched tasks across three seeds.

## Spend

Metered OpenAI arms in single-digit dollars; the Anthropic arm ran on a flat subscription.
The one billing event worth recording: the metered quota exhaustion that halted a run
mid-experiment, which is itself evidence for the pricing-regime ruling.

## Post-publication measurement

Lab 012 re-ran the held-out fifth arm on the metered API with per-task usage capture: 22 of
22 again, $13.79, 10.96 million tokens. And the separation came back against the tier: a
mini model in the same harness matched all 22 for $3.14. The harness was the lever. Both
labs stand.

## What this hands lab 012

One question, clean: hold Claude Code fixed, drop in a less capable model. If a weaker model
in the same agentic harness still clears the tail, the harness was the lever. If recovery
collapses toward the loop numbers, the tier was carrying it. The 22-miss pool, the gate, and
the seat discipline travel forward unchanged.