Buy the harness, not the tier

Measured on localized repair with an executable test, one task pool of 22, one week of vendor pricing. The rulings are about where capability enters an agentic stack and what each rung of an escalation ladder costs per verified unit of output. The open edges are repository-scale debugging without localization, domains without an executable evaluator, and variance bounds beyond two runs per arm.

Self-funded editorial. No vendor paid for this answer, reviewed it, or saw it before publication. The free local worker is Gemma, a Google model, run on owned NVIDIA hardware; the paid arms were metered OpenAI and Anthropic calls, and the rented-GPU arms ran on AWS. Disclosure: Google Cloud is a client of The CTO Advisor LLC; NVIDIA, OpenAI, Anthropic, and AWS are not. The ruling is the author's alone.

An architect escalating repair work has rungs available: a free local loop, a paid mid-tier model in that same loop, then a frontier model inside an agentic harness. Which rung is actually load-bearing, and what does each one cost? Lab 011 ended on that question. It showed a held-out run where Claude Code cleared everything, but that run swapped the harness and the model at once, so it proved neither. This lab was built to answer the one question that run left standing, and the answer reframed the whole ladder.

The bench: the 22 tasks Lab 011's free local loop first missed, each a real bug-fix pull request shipping its own test suite as an unfalsifiable pass/fail gate. Fourteen arms across two apparatuses, the constrained repair loop from Lab 011 and Claude Code driven headless through a translation proxy. Six models, from Gemma 4 26B and 31B on owned hardware, through hosted gpt-5-mini and gpt-5.4-mini, to Claude Opus 4.8 and Opus 5. Three arms were replicated, and every cost figure is directly measured: the mini arm from a dedicated API key's dashboard line, the Opus arms from per-task usage envelopes on the native API path. The rented-GPU arms ran on AWS. Total spend was about $121: $73.75 of rented infrastructure and $47 and change in metered tokens, including $5.45 killed by a mid-run credit exhaustion and roughly $24 of GPU left idling overnight. The mistakes stay in the bill because a reproducer pays for theirs too.

Start on capacity you already carry. A Gemma 4 31B in the constrained loop clears 17 of the 22 on deterministic test feedback alone, and the gate hands back the exact five it couldn't solve. The earlier Xeon lab priced that rung: about nine cents per verified repair on rented Granite Rapids, six at the measured Savings Plan rate, and approaching zero on owned idle hardware or inside a commitment you already bought and didn't use. This isn't a story about owning a Spark. Unused cloud-spend commitment is idle capacity too, and so is the headroom left in a flat-rate subscription seat. Three forms, one category.

The money is not in the tier

The next rung up is the one that doesn't work. A paid mid-tier model in that same constrained loop scored 16 of 22, and 17 on a second run, against the free local model's 17 and 17. Two runs each, dead parity. Buying a better model without changing the apparatus around it bought no separation from free. That result reframes the ladder, because it means the money is not in the tier.

It's in the harness. The same mini that plateaued at 16 and 17 in the loop clears all 22 inside Claude Code, for $3.14. Six tasks recovered from apparatus, not parameters, and the apparatus isn't exotic: read the repository, run the tests, read the failure, try again, with tools instead of a fixed protocol. Four configurations cleared all 22, gpt-5.4-mini twice, Opus 4.8, and Opus 5, all inside the same agentic harness. The Opus arms finished the identical work for $13.79 and $12.56, four times the mini's bill, which per verified repair is $0.14 against $0.63. And if you escalate only the five the gate names instead of pushing all 22 through a frontier model, the residual costs $1.12 against $12.56, an eleven-fold spread for the same finished work.

One constraint keeps you honest, and it's why the escalation rung has to be bought rather than substituted. You can't simply wrap the free local model in the harness and skip the paid step: Gemma 4 31B goes from 17 down to 11 inside Claude Code. The harness multiplies a model that can operate its instruments and taxes one that cannot. Instrument literacy is its own capability, and it doesn't track parameter count. Above that bar the tier is close to irrelevant, and below it the harness makes things worse. Where the bar actually sits is open: Gemma 4 31B fails it and gpt-5.4-mini clears it, which brackets the bar but doesn't locate it. Finding the cheapest self-hostable model that still multiplies is the interesting next descent for anyone with idle VRAM.

You cannot rent your way there

So should you buy the GPU class you can actually get, and self-host the escalation rung? I priced it, and no. The rented arm cleared 9 of 22 at $4 to $6 of machine time: more than the API, for under half the capability. The reason is structural. Agentic sessions are bought with memory, not compute. Each one parks its context in VRAM for its whole life while barely touching the die, so the honest unit is dollars per concurrent session per hour. Four entry instances give about nineteen sessions for $7.44 an hour. The box already on my bench holds fifteen for nothing. Renting the liquid class buys four more sessions at seven dollars an hour, which is not a purchase, it's a rounding error with a meter attached. If you do rent, scale out with the smallest instance rather than up: consolidated multi-GPU boxes charge you for interconnect that independent sessions never touch.

What separates the rungs is what a retry costs. On idle capacity a retry is free, so a variance-prone cheap model isn't a liability, it's a method: let it grind and let the gate certify each attempt. The Xeon lab audited 585 scored attempts and found no validator pass that failed its held-out checks, which is what makes grinding safe rather than reckless. On a metered API the arithmetic inverts. Every retry bills, so what you're actually buying up the ladder is first-pass success. That's instrument literacy, seen from the invoice.

The number that survived

Almost every number I believed at the start of this lab died during it. The biggest casualty was the thesis the first draft of this analysis was built on: token efficiency. Real usage capture dissolved it. Three models spanning a fourfold price range landed within 13 percent of each other on tokens, and the mini run twice against itself varied 24 percent, so the gap between models is smaller than the noise inside one of them. The harness sets the token budget. The tier multiplies the rate. That's AI Factory Economics doing exactly what the framework warns about, naming token fixation as a trap, and I walked straight into it before the measurement pulled me out. The denominator that survived is cost per gate-verified repair, and it tracked everything the tokens tracked nothing of.

The fair objection: 22 tasks, mostly single runs, isn't tier irrelevance exactly what noise produces? The claim survives its own noise measurement, which is more than most tier comparisons attempt. The replicated arms calibrate the rest: solve counts hold within a task of themselves, individual borderline tasks flip freely, and token comparisons need error bars 24 percent wide. And the pool is conditioned deliberately. These are the 22 repairs a free local model couldn't clear on its first pass, the population that actually reaches an escalation decision. What's not proved: the subscription rung's near-zero marginal cost is structural arithmetic, no arm ran under subscription auth. Every task arrived with its files named, so repository-scale debugging without localization is open. And a one-task probe of Opus 5 extrapolated to a $19 to $20 arm the measured arm finished at $12.56. Single-task probes establish pricing and nothing else.

The transferable rule isn't a ladder. It's a dispatcher. Spend committed capacity first at every rung it exists: idle hardware, unused cloud commitment, unused subscription headroom. Escalate to metered capability only for what the gate hands back, buy the cheapest model above the instrument-literacy bar, and route overflow back down when the budget binds instead of the clock. The gate makes both directions safe, because a verified repair is verified regardless of which rung produced it, and the determines-done authority never moves. Buy the harness, not the tier. Then define your unit of verified output before you argue about models, because without that denominator, every tier debate is unfalsifiable.

The numbers

Where each layer belongs

Assessments at the time of the lab

Method and disclosure

Self-funded editorial. The harness is loopcontrolbench, published open under the MIT license at github.com/kltownsend/loopcontrolbench: the reproduce-or-drop gate, the sandboxed runner, and the arm definitions, including the Claude Code driver and the translation-proxy configuration added for this lab. The vendor assessments and the layer placements this evidence feeds stay proprietary.

This lab tests an existing model rather than proposing one. The AI Factory Economics Framework was published at thectoadvisor.com/blog/2026/01/07/from-ai-factory-metaphor-to-cio-grade-economics/, and every cost figure here is denominated in that framework's unit of business output: one gate-verified repair. The framework names token fixation as a failure mode; the first version of this analysis committed it, and the correction is reported in the writeup rather than quietly removed.

The essay on this page was rendered from the lab’s frozen structured record by a model writing under this practice’s voice specification, with a deterministic gate checking every figure against the record before publication. The record is the canonical surface: where prose and record disagree, the record rules.