# Lab detail — Trained by whom (you can't price the task without pricing the fine-tune)

Raw lab detail for labs.layer2c.com/labs/trained-by-whom. A one-week campaign (2026-08-17 to
2026-08-23) on two owned NVIDIA DGX Sparks, one rented AMD MI300X, and four hosted model APIs,
testing whether an owner-side fine-tune improves a per-task agentic code-repair process over the
base model, with cost per gate-verified repair as the denominator.

**What ships:** the decision procedure, the measured results, the cost model, the corrections,
the failure-mode catalog, and the open-source instrument (loopcontrolbench, MIT). **What stays
proprietary:** the specific 22-task set and its extracted tests, the teacher trajectory corpus,
the tuning particulars, and any vendor-assessment consequences. Returns, not the algorithm.

## The question

The corpus held two fine-tuning claims that had never met. An earlier lab measured an owner's
LoRA moving accepted output from 23% to 70% on a bounded single-shot task and ruled: own the
weights. A later lab measured the per-task agentic regime and found the separator is training,
not size, and that the training was the vendor's. This lab ran the missing arm: does an owner's
fine-tune improve a gate-verified agentic repair process? The measurement kept stalling on a cost
the original framing had minimized, the cost of producing the model at all, and the deliverable
became a decision procedure for when fine-tuning, and therefore self-hosting, is the right lever.

## Setup

The eval instrument is a fixed pool of 22 localized repair tasks, each with an executable test,
scored by a deterministic reproduce-or-drop gate unchanged across five prior labs. New apparatus:
an 82-task disjoint training pool mined from a dozen fresh repositories, a teacher trajectory
corpus, a synthetic precision-skill factory, and a self-generated harvest of gate-passed loop
solutions. The tuning subject was one open model family. Comparators were tool-trained open peers
on identical serving, a mini-class hosted model as the frontier reference, and an open
frontier-scale model served on owned clustered hardware.

Two agentic surfaces were tested. The **harness** drives the model through a tool channel with
exact-match edits, pricing edit precision under long-context load. The **loop** regenerates
whole-artifact edits judged by the gate, exercising diagnosis and generation in clean context
every turn.

## Corrections (published before the conclusions that depend on them)

**The harness "wall" was substantially a serving bug.** The first-phase harness numbers, a base
edit-land rate near 37% against tool-trained peers near 96%, were measured with a tool-call
parser that silently dropped the model's native emission format, scoring correctly-shaped tool
calls as plain text. Served with the correct parser, the untuned base scored 15 of 22 where the
wrong parser had it at six to eleven. A clean cross-model re-measurement was not finished before
the rented card came down, so the execution-deficit reading is reopened, not overturned. The
serving contract is part of the model.

**The consolidation tune ran; an earlier draft said it would not.** The best-designed owner tune,
a self-consolidation adapter trained on the model's own gate-passed loop solutions, was built and
evaluated: 14 of 22 in the loop against a fair untimed loop base of 12. On 22 tasks that two-task
delta sits inside sampling noise.

**A harvest workspace bug inflated an early miss rate.** The solution harvester built prompts
before resetting a workspace shared with harness runs, so on nodes that had run the harness the
model was shown pre-edited files and its edits failed to apply. Corrected, the base first-shot
keep rate on the training pool was roughly 48%, not the ~12% first observed.

## Measured results

- **Owner fine-tunes, gate-verified gain.** Three reachable families (imitation from a teacher,
  isolated-skill precision) produced zero gate-verified improvement; the imitation tune was
  actively destructive, transferring the teacher's brevity without its precision. The
  self-consolidation adapter nudged the loop count by two tasks, inside noise.
- **The frontier reference.** A mini-class hosted model solved all 22 tasks for about $3.14 at a
  ten-turn median.
- **Apparatus fit.** The constrained loop clears the solvable set deterministically at a fraction
  of the harness token cost. The loop is the matched instrument for an untooled open checkpoint,
  not the lesser rung.
- **The exception.** An open frontier-scale model, served on two clustered owned Sparks, swept
  the pool twice at a seven-minute median for electricity and hours, no fine-tune. Self-hosting
  succeeded where fine-tuning did not, on off-the-shelf open weights.
- **Serving variance.** Back-to-back same-config hosted runs scored 6 and 11; cache warmth
  between replicates is the standing suspect. Single-run cell numbers from shared-cache venues
  are weather.

## The cost model (the part the pitch omits)

Cost per task is unknowable until the fine-tune is priced, and the fine-tune is capital that
amortizes, not an implementation detail. Fine-tuning is therefore volume economics: a fixed cost
divided across every task it solves, catastrophic over one task, invisible over a million of one
shape.

The search itself was priced. Metered compute was about $73 across roughly 90 owned GPU-hours,
but compute is the floor, not the bill. The dominant cost is attended engineer time and calendar,
and the two are not the same clock: guards, progress beacons, and overnight runs decoupled the
labor from the wall time. The **cheat math**: compress the search to a single workweek at rented
MI300X speed (at most 40 GPU-hours against ~90 on the slower owned box) and the blended resource
rate is $103 to $106 an hour, of which the GPU is $3 and the engineer is ~$100. Compute is three
percent of the hour. Serving cost is the last branch, below yield, apparatus fit, what the vendor
trained, and whether training was needed at all.

## Key findings — the decision procedure

Self-hosting and fine-tuning are different decisions the pitch fuses. **Compliance is an override,
not a condition:** where data cannot leave the VPC or the workload is regulated or air-gapped,
self-hosting is mandatory regardless of capability. Off-the-shelf open weights can win self-hosting
with no tune. Fine-tuning is the narrowest door, and it splits into capability tuning and cost
tuning (distillation), which do not share a rule.

Capability fine-tuning pays only when five conditions hold together:

1. The foundation model fails your **gate**, not just your **budget**. Acceptable output at any
   price is a cost problem, not a capability problem.
2. The missing capability is **behavior, not knowledge**. Retrieval injects what a model knows;
   fine-tuning changes how it acts. If a corpus plus a frontier model clears the gate, it was a
   retrieval problem in a fine-tune costume.
3. A **deterministic gate** exists. No gate, no cost-per-verified-task, no economics. Most domains
   lack this.
4. **Value per task is high and volume is large** enough to amortize a fixed cost.
5. You have **priced the whole how**, including the deterministic enforcement the tune does not
   remove. A fine-tune moves behavior most of the way; a hard invariant still leaks the base
   prior, so a deterministic layer enforces the last mile, and that enforcement is a recurring
   cost that never amortizes. Owning the model does not retire the validator; it shifts work onto
   it.

Code repair meets conditions 3 and 4 and fails condition 1: the frontier already clears the gate
cheaply. Most enterprise domains that look like fine-tuning candidates are retrieval problems.

## Failure-mode catalog (operator error, now encoded as gates)

Roughly a quarter of the campaign's spend was avoidable operator error, each fix now a mechanical
gate in the instrument:

- A proxy routing to the wrong API surface, and a served context cap below the reference envelope,
  each silently invalidating an arm. Fix: a venue preflight (envelope check, arm-scale prefill
  probe, end-to-end tool smoke, provenance stamp) that refuses a bad launch.
- A tool-call parser mismatch that buried a working model. Fix: the parser is part of the serving
  contract and is verified before scoring.
- A model alias that resolved to the base weights instead of the adapter. Fix: routing is verified
  by observing the adapter compute path, not by a provenance stamp that cannot tell them apart.
- Two host reboots from pushing unified memory past its wall during training. Fix: chunk every
  long forward, and canary any new sequence length on one node before committing the pair.

## What did NOT get settled (honesty)

- It did not prove owner fine-tuning never pays. It proved three reachable families bought nothing
  and the best fourth nudged the count within noise, on one domain that fails the decision test.
  The bounded single-shot regime where an earlier lab measured a 3x owner gain stands untouched.
- The five-condition procedure is validated against a domain that fails it, not one that passes.
  Its predictive value on a passing domain awaits the next lab, and the hard part there is
  building the gate.
- The parser correction reopens the phase-one harness numbers rather than cleanly replacing them.
- The consolidation tune's harness arm was torn down before completion as immaterial to the
  conclusion.
- The distillation branch and the compliance override are named and reasoned, not measured here.