Lab 018 · Editorial lab

You can’t price the task without pricing the fine-tune.

By Keith Townsend · August 23, 2026

We set out to measure cost per gate-verified task, own hardware against hosted API, and answer whether an expensive owned box earns its price. The measurement kept stalling on a cost we had minimized: the cost of the model itself. You do not get the model for free. You build datasets, run a tune, evaluate it, throw it away, and try again, and the infrastructure the discarded models burned does not vanish because the models did. Pricing a task means pricing that search. This lab ran the search to the end on one domain and came back not with a number but with the procedure that decides whether the number is worth computing at all.

Measured on localized repair with an executable test: one 22-task pool, one open-model family as the tuning subject, one week of checkpoints and prices. The decision procedure is the deliverable; the code-repair verdict is the worked example that produced it. The open edges are a domain that satisfies all five conditions, repository-scale work without localization, the file-geometry wall, and a clean re-measurement of harness edit-landing once the tool-call parser was corrected.

Do settle compliance first, then run the five-condition test. Compliance is an override, not a condition: if the data cannot leave the VPC, or the workload is regulated or air-gapped, you self-host regardless of capability and the conditions below do not apply. Where the hosted frontier is a legal option, the test gates capability fine-tuning: the model has to fail your gate not your budget, the missing capability has to be behavior a corpus cannot inject, a deterministic gate has to exist, the value and volume have to amortize a fixed cost, and you have to price the whole how including the enforcement the tune does not remove. Cost tuning to distill a cheaper model that matches an expensive one is a separate branch that lives in the volume math, not this test.

Don’t treat the fine-tune as an implementation detail inside a cost-per-task number. It is capital. It amortizes across every task it solves, which makes fine-tuning volume economics: catastrophic over one task, invisible over a million of one shape. Any cost model that expenses the training run and forgets the search that preceded it, and the scrap the search burned, is pricing the wrong thing. This search cost roughly $73 and ninety GPU-hours to buy a two-task gain that a hosted model beats outright.

Don’t assume owning the weights buys you a corpus’s worth of knowledge. Retrieval injects what a model knows. Fine-tuning changes how it acts. If a corpus plus a frontier model clears your gate, the task was a RAG problem wearing a fine-tune costume, and most enterprise domains that look proprietary are exactly that. Fine-tuning’s defensible territory is behavior a corpus cannot supply, which is a narrow intersection, not a default.

Do expect the deterministic gate to survive the tune, and budget it. The fine-tune moves behavior most of the way and a hard invariant still leaks the base model’s prior. This practice’s voice work is the proof: even fine-tuned and prompt-guarded, the model reverts to em-dashes, and only deterministic tooling in the loop enforces the pattern. That gate is a recurring cost that never amortizes. Owning the model does not retire the validator. It shifts work onto it.

Do match the apparatus to the model and get the serving contract right before you read a single score. In the loop the consolidation tune reached 14 of 22 against a fair untimed base of 12, a nudge inside noise. On the harness the story was a serving bug: the wrong tool-call parser silently dropped the model’s native emission, and corrected, the untuned base scored 15 where it had shown six to eleven. Loop and harness are different apparatus, so do not read the loop’s 14 against the harness’s 15. Serving correctness is part of the model contract, and a mechanical preflight, envelope, arm-scale prefill, end-to-end tool smoke, provenance stamp, is now the gate every arm passes first.

Self-funded editorial. Disclosure: Google Cloud is a client of The CTO Advisor LLC, and Google models sit at the center of this lab as both tuning subject (Gemma 4) and benched baselines (Gemini 2.5 and 3.6 Flash, Vertex). Anthropic’s Claude Code is the primary harness and is itself implicated in the findings. Hot Aisle, AMD, NVIDIA, OpenAI, Mistral, Alibaba, and DeepSeek are not clients. No vendor commissioned, funded, or previewed any of this.

The lab set out to price a task and discovered it could not, because the model is not free and the fine-tune that produces it is capital, not an implementation detail. The deliverable is the decision procedure that comes before the price: five conditions that have to hold together before fine-tuning, and therefore self-hosting, is the right lever. The foundation model has to fail your gate rather than your budget. The missing capability has to be behavior a retrieval corpus cannot inject. A deterministic gate has to exist. The value per task has to be high and the volume large enough to amortize a fixed cost. And you have to price the whole how, including the deterministic enforcement the tune does not remove.

Code repair was the worked example, and it fails the test on the first condition. A mini-class hosted model solves the whole 22-task pool for $3.14. Three owner-side tunes, imitation and isolated-skill, produced no gate-verified improvement, and the imitation tune was actively destructive. The fourth and best-designed tune, a self-consolidation LoRA trained on the model’s own gate-passed loop solutions, did run: 14 of 22 in the loop against a fair untimed loop base of 12, a two-task delta that sits inside sampling noise on a pool this size. A tune that at best nudges the count and still loses outright to the frontier is the cleanest proof that the domain belongs to the foundation model even when ownership works.

Two corrections sharpened the picture. The harness “mechanical wall” that phase one read as a model deficit was substantially a serving bug: the tool-call parser was dropping Gemma’s native emission, and with the correct parser the untuned base scored 15 of 22 instead of six to eleven. The serving contract is part of the model, and getting it wrong reads as incapacity. And the fine-tune never retired the validator: the practice’s own voice work shows a fine-tuned, prompt-guarded model still reverting to em-dashes until deterministic tooling enforces the pattern. The how is layered, and the bottom layer is a gate that never amortizes.

The economics generalize as amortization and branch order. A fine-tune is a fixed cost divided across every task it solves, so fine-tuning is volume economics: catastrophic over one task, invisible over a million of one shape. Build or buy does not change the term, only its shape, and the buy path carries a transfer risk the build path does not, measured here when a community tune worked only inside its own scaffold. Serving cost, the branch the original spec opened on, is the last branch, not the first: above it sit yield, apparatus fit, whether the vendor trained the capability, and whether you needed to train at all or could have retrieved. And self-hosting is not the same decision as fine-tuning: compliance forces it above any capability question, and off-the-shelf open weights can win it with no tune at all, which is why the frontier-owns-code verdict is a verdict about fine-tuning, not about owning hardware.

The campaign priced its own search, which is the exhibit the fine-tuning pitch never includes. Roughly $73 metered, ninety owned GPU-hours, six days, and two host reboots to buy a two-task nudge inside noise, about a quarter of it avoidable operator error now encoded as mechanical gates. Priced the way a business incurs it, compressed to a single workweek at rented-MI300X speed, the resource cost is around $105 an hour, of which the GPU is $3 and the engineer is the other hundred. Discovery is the expensive part, and it is expensive even when it is disciplined. The number a self-hosting business case needs is not the training run. It is the search, and the search is paid in attended hours, not dollars of electricity.

The detail

The lab began as a cost-per-task measurement: run one agentic repair pool against owned hardware and against hosted APIs, and find what a solved task actually costs each way. The measurement kept stalling on a term the original model had minimized. You do not get the model for free. Testing whether an owner’s fine-tune beats a foundation model means building the fine-tune, and building it means datasets, a training run, an evaluation, a disappointing result, a data change, another run, and a pile of discarded checkpoints whose infrastructure cost did not vanish when the checkpoints did. Pricing the task means pricing that search. So the deliverable stopped being a number and became the procedure that decides whether the number is worth computing: five conditions that have to hold together before fine-tuning, and therefore self-hosting, is the right lever.

Code repair was the worked example, and it fails on the first condition. A mini-class hosted model solved the entire 22-task pool for $3.14 in forty minutes, which means the foundation model fails nobody’s gate here, only a budget the frontier already undercuts. Against that floor the owner tunes ran. Three of them, imitation from a teacher and an isolated-skill precision factory, produced no gate-verified improvement, and the imitation tune was worse than nothing: it transferred the teacher’s brevity without the precision that makes brevity work and damaged behaviors the base model already had. The fourth was the one the arithmetic said to build: a self-consolidation LoRA trained on the model’s own gate-passed loop solutions, harvested from a disjoint pool. It ran. Fourteen of 22 in the loop against a fair untimed loop base of 12. On twenty-two tasks a two-task delta sits inside sampling noise, so call it what it is: a small nudge, not a breakthrough, measured against the loop’s own base and not against the harness. A tune that at best nudges the count and still loses outright to a $3.14 hosted run is the cleanest proof that the domain belongs to the foundation model even when ownership is doing its job.

Two corrections kept the read honest. The harness failure that phase one had called a model deficit, a 37% edit-land rate against tool-trained peers near 96%, was substantially a serving bug. The tool-call parser was dropping Gemma’s native emission format, scoring correctly-shaped tool calls as plain text. With the correct parser the untuned base scored 15 of 22 where the wrong one had it at six to eleven. That does not erase the peer gap, and a clean cross-model re-measurement was not finished, but it relocates a chunk of the wall from the model to the serving contract, which is itself part of the model. Two apparatus, two numbers: the loop’s 14 and the harness’s 15 are different instruments and do not compare to each other. The second correction is that the tune never retired the validator, and the parser is the in-domain proof of it: a deterministic layer had to decode what the model would not reliably emit before the harness could score it. The same pattern shows in this practice’s voice work, where a fine-tuned, prompt-guarded model still reaches for the em-dashes a gate has to strip. The how is layered. The fine-tune moves behavior most of the way, and a deterministic layer holds the invariant the base prior keeps violating.

From there the economics generalize, and they generalize as amortization. A fine-tune is a fixed cost divided across every task it solves. Over one task it is catastrophic; over a million tasks of one shape it rounds to zero. Fine-tuning is volume economics, and code repair fails on volume as surely as on the frontier price: even a free-to-run tune has to amortize its build against a domain where the frontier already wins per task. Buying the tune instead of building it does not zero the term, it reshapes it. A community adapter is a free download and a real bill in GPU-hours to stand up, and it carries a transfer risk the build path does not, measured here when a well-regarded community tune worked only inside its own training scaffold and nowhere else. Either way you pay to learn whether the weights clear your gate.

The capability question sits underneath all of it, and it is the one most enterprises get backward. Retrieval injects what a model knows. Fine-tuning changes how it acts. If a corpus plus a frontier model clears the gate, the task was a retrieval problem wearing a fine-tune costume, and the proprietary-knowledge domains that look like fine-tuning candidates are usually exactly that. Fine-tuning earns its cost only where the missing thing is behavior a corpus cannot supply, the gate rejects the frontier, the volume amortizes the build, and the value justifies the search. That intersection is small, which is the finding, not a disappointment: real fine-tuning use cases are rarer than the market implies. And the arithmetic for self-hosting falls out of the same order, once you stop fusing it with fine-tuning. Self-hosting is forced by compliance regardless of capability, no amount of frontier quality overrides a workload that cannot leave the VPC. It can also be won outright by off-the-shelf open weights that need no tune, the way an open model swept this pool self-hosted and untrained. Fine-tuning is only the narrowest door into it. Below all of that, serving cost is the last branch: above it sit yield, apparatus fit, whether the vendor trained the capability you need, and whether you needed to train at all. Own the model when compliance demands it, when open weights already win, or when the five conditions hold and the volume is yours. Rent the frontier otherwise, which is more often than the pitch admits. And price the search in the currency that dominates it, which is not GPU-hours but attended engineer time and calendar, the terms the owned box hides and the CFO does not.

You just tuned badly, and your best tune only gained two tasks. Better data or RL closes it.

The best-designed tune did run, and it worked: a self-consolidation LoRA trained on the model’s own gate-passed loop solutions scored 14 of 22 in the loop against a fair untimed loop base of 12, a two-task delta that sits inside sampling noise on a pool this size, zero regressions. That is the best the reachable families produced as we reached them, and it is the point. A nudge over base still loses outright to a mini-class hosted model that solves all 22 for $3.14 in forty minutes. The tune succeeding does not move the domain; it confirms the domain belongs to the foundation model even when the tune helps.

The families an owner can reach are imitation, isolated-skill, and consolidation, and all three are measured here. What remains is frontier-scale RL post-training, which an owner cannot buy as a process, only as a product. That product sits in the open-weights catalog: Devstral and Qwen3-Coder land their edits where Gemma’s family does not, and DeepSeek V4 Flash swept this pool twice on owned clustered hardware for nothing but memory and hours. Acquiring the capability by model choice costs nothing. Acquiring it by training cost this lab a week to establish that, for this domain, it should not have been attempted, and the enterprise version runs the same shape at a thousand times the bill.

What this opened

What domain satisfies all five conditions? Code repair fails on the first: a hosted model clears the gate for pocket change. The domain where fine-tuning pays needs the foundation model to fail the gate, the missing capability to be behavior a corpus cannot inject, a deterministic gate to exist, and the volume to amortize the build. The Articul8 sponsored work named the Layer 2C reasoning plane but lacks a workload to test at this task-level economics. Finding one, and building its gate before any GPU spend, is the next lab. The gate is the hard part; no gate, no economics.

How far does the parser correction reach? The harness numbers phase one read as a model deficit were measured with a tool-call parser that dropped Gemma’s native emission. Corrected, the untuned base jumps from six to eleven up to 15 of 22. The tool-trained peers were parsed correctly, so the 37%-versus-96% edit-land comparison was partly apples to oranges. A clean re-measurement of edit-landing across every model with the correct parser was not completed before the rented card came down. Until it is, treat the phase-one execution-deficit finding as reopened, not overturned.

Does the dispatcher survive losing localization? Every task here arrived with its files named, which switches off the one advantage agentic scaffolding is priced for. Repository-scale fault localization is the harness’s home game and the loop cannot attempt it. Whether triage-then-escalate holds, and where the local rung’s share falls, when the pool withholds file names is the next pool to mine, and the domain hunt above may find its gate there.

Whose harness serves whose weights? Google’s Antigravity was probed as the bring-your-own-model path for Gemma and closed as unreachable: the desktop app silently declines a local endpoint, and the CLI, inspected at the binary level, lists hosted models only, carries a Google-and-Vertex provider enum with no OpenAI-compatible entry, and gates custom models behind a server-side experiment flag. Local weights under Google’s harness are not routable by any visible surface. The managed floor has moved up into the harness layer, and a routing check that counts requests at the serving log is the mandatory gate for any bring-your-own-model claim.

What it did not prove

  • It did not prove owner fine-tuning never pays. It proved that on localized code repair, with one open-model family as subject, three reachable families bought nothing and the best-designed fourth nudged the count by two tasks, a delta inside sampling noise on 22 tasks, measured on one loop configuration without multi-seed runs or confidence intervals. The decision procedure is the transferable claim; the code verdict is one instance of it. The bounded-generation regime where Lab 002 measured a 3x owner-LoRA gain stands untouched, and is itself an instance where the five conditions plausibly hold.
  • The parser correction reopens the phase-one harness findings rather than cleanly replacing them. The untuned base scores 15 of 22 with the correct tool parser, but a full re-measurement of edit-landing across all models on the corrected serving was not finished before the card came down. The execution-deficit story is reopened, not overturned.
  • The consolidation tune’s two-task gain is measured in the loop against a fair untimed control. Its harness arm was torn down before completion as immaterial to the conclusion, so the tune’s effect on the tool-driven surface is only partially observed.
  • The five-condition procedure is validated against one domain that fails it, not against a domain that passes. Its predictive value on a passing domain, the case where fine-tuning is supposed to win, is asserted from mechanism and awaits the next lab. The procedure also gates capability fine-tuning only; the distillation branch (a cheaper model tuned to match an expensive one at volume) is named but not measured here, and the compliance override that precedes all of it is a constraint this lab asserts, not one it tested.
  • The DeepSeek V4 Flash sweep is this pool only, on a two-box cluster this practice already owned, and whether it survives an un-localized pool or a different task class is unmeasured.
  • Roughly a quarter of the campaign’s spend was avoidable operator error, including two host reboots from pushing unified memory past its wall. The gates built from those errors are asserted to prevent recurrence from mechanism, not yet measured across a subsequent campaign.

Notes from the lab

The finding moved twice under me, and both moves were the same move: stop optimizing the model and start pricing the decision. The first time, the harness kept losing to the loop and the reflex was to train the model up to the harness. The question that mattered was why we were chasing the harness at all when the numbers already said the apparatus should fit the model. The second time, the whole exercise looked like a failure to make a fine-tune win, until the failure resolved into the thing worth publishing: not that the tune lost, but the procedure for knowing in advance that it would.

The parser correction is the humbling one. A wall we had attributed to the model for a week was substantially a wrong tool-call parser, deterministic tooling we controlled, silently dropping correctly-shaped output. It is the lab’s own thesis turned on the operator: the deterministic layer around the model is load-bearing, and getting it wrong reads exactly like the model being incapable. The serving contract is part of the model, and I had not been treating it that way.

What I keep is the bill and the shape of it, and the shape is the correction I owe. The metered number is roughly $73 and the compute is ninety GPU-hours, but that is the cheap part and citing it alone understates the truth. The dominant cost is my time and the calendar, six days of it, and my time is not free at any rate a CFO would recognize. The honest twist is that ninety GPU-hours was never ninety hours of me at the keyboard: guards and beacons and overnight runs kept the boxes working while I did not, which is the one lever that keeps the human bill down. The cheat math puts a number on it. Compress the whole search into a single workweek at rented-MI300X speed, at most forty GPU-hours instead of ninety on the slower owned box, and the blended resource rate is $103 to $106 an hour, of which three dollars is the GPU and a hundred is me. Compute is three percent of the hour. The pitch for owning your model prices the training run. It never prices the search, and the search is priced in attended hours and days, not in dollars of electricity.

The numbers

The frontier the tune must beat
gpt-5.4-mini, 10-turn median; the whole pool solved for the price of a coffee, which is the first-condition failure for code repair
mini 22/22 · $3.14 · 40 min
Owner fine-tunes, gate-verified gain
imitation and isolated-skill: zero; self-consolidation LoRA: 14/22 vs a fair untimed loop base of 12/22, a two-task delta inside sampling noise on 22 tasks, and it still loses to the frontier
3 tunes · 0 · then +2 (noise)
Harness wall was a serving bug
the wrong tool-call parser silently dropped Gemma’s native emission; the gemma4 parser recovered it; the serving contract is part of the model
base 6-11 → 15/22
The dispatcher, per verified repair
loop clears 17/22 deterministically; mini 22/22; DeepSeek V4 Flash 22/22 twice, 7-min median, on two clustered Sparks
loop ~$0 · mini $0.14 · DSV4 $0
Fine-tuning is volume economics
catastrophic over one task, invisible over a million of one shape; the term that was invisible while we tested a domain the frontier owns
fixed ÷ tasks solved
The how is layered
fine-tune moves behavior most of the way; a hard invariant leaks the base prior (em-dashes) until a deterministic gate enforces it, and that gate is recurring cost
tune + non-amortizing gate
Build vs buy does not zero the term
DeepSeek community tune: free download, real GPU-hours to stand up, and it worked only inside its own training scaffold
validation + transfer risk
Cost of the search, cheat math
compress to a single workweek at rented-MI300X speed (≤40 GPU-h vs ~90 on the owned Spark); blended rate $103-106/hour, of which the GPU is $3 and the engineer is ~$100; ~40h × ~$105 ≈ $4,100-4,250. Compute is 3% of the hour; the human is the bill. Metered compute alone was ~$73
~$4,100 · one workweek · $105/hr blended

Where each layer belongs

LayerPlacement
Layer 0 · Compute
Compute & Network Fabric
Two altitudes, one placement. The owned boxes carried training, serving, loops, and diagnosis at zero marginal dollars, including a two-Spark DDP run once the fabric proved bandwidth-bound rather than latency-bound. The cluster’s highest use surfaced late: unified memory holds an open frontier-scale model that sweeps this pool. Capacity, not capability, is what ownership buys, and a host reboot twice was the price of pushing memory past its wall.
Retained
Layer 2B · Runtime
Application Runtime & Execution
The deterministic layer is the part that never regressed and the part the whole finding rests on: gate, loop, crash-safe results, a venue preflight that refuses bad launches, and, this campaign, a tool-call parser correction that recovered a model the wrong parser had buried. Every lesson that lived as code survived; every lesson that lived as agent memory was repaid this week. The runbook is the control plane; agent memory is a cache.
Retained
Layer 2C · Reasoning
Agentic Infrastructure — The Reasoning Plane
The repair-reasoning seat stays shoppable, and this lab prices it. An owner controls the apparatus and the serving conditions for free and the vendor’s post-training only through model selection. The reachable tune bought two tasks and still lost to the frontier on cost. Retain the seat by choosing tool-trained weights or matching the apparatus to untooled ones, and fine-tune only where the five conditions hold. The determines-done authority never moved.
Retained via selection, and priced

Method and disclosure

Self-funded editorial. The instrument is loopcontrolbench, open under MIT: the reproduce-or-drop gate, the sandboxed runner, both harness drivers, and the additions this campaign contributed, a mechanical venue preflight (envelope, arm-scale prefill, end-to-end tool smoke, provenance stamp), the trajectory taxonomy analyzer, the land-rate gate, the synthetic precision factory, the loop-solution harvester, and a two-Spark DDP training path. The teacher corpus, the trained adapters, and any vendor-assessment consequences stay proprietary. Cross-lab operating discipline accumulated into specs/BENCH-METHOD.md, including the rules this campaign paid for: the tune must earn its place against a clean baseline, serve at the reference envelope with the correct tool-call parser, verify routing by counting requests, cap the loop at two turns, and kill by PID never by pattern.

Every invalidated arm is archived with its defect named, not deleted: a proxy that routed to the wrong API surface, a context cap below the reference envelope, a ceiling miscalibrated for contention, and a tool-call parser that dropped the model’s native emission. Serving configuration travels with every result via the venue note and a per-task backend probe. The comparisons are kept distinct by design: base versus tuned is the measurement, tuned versus teacher is distillation fidelity, and local versus managed serving is the economics, which this campaign resolved one branch above where the spec opened it, and then generalized into the decision procedure that is the lab’s finding.

Download the raw lab detail (Markdown)