{ "slug": "trained-by-whom", "lab_number": 18, "title": "You can’t price the task without pricing the fine-tune.", "type": "editorial", "status": "published", "sponsor": null, "author": "Keith Townsend", "date": "August 23, 2026", "date_iso": "2026-08-23", "layers": [ "layer0", "layer2b", "layer2c" ], "axes": [ { "instrument": "4plus1", "layers": [ "layer0", "layer2b", "layer2c" ], "has_dapm_table": true } ], "supersedes": [], "superseded_by": null, "corrects": [], "corrected_by": [], "vendors": [ { "key": "google", "role": "model" }, { "key": "nvidia", "role": "hw" }, { "key": "amd", "role": "hw" }, { "key": "hotaisle", "role": "cloud" }, { "key": "anthropic", "role": "harness" }, { "key": "openai", "role": "model" }, { "key": "mistral", "role": "model" }, { "key": "alibaba", "role": "model" }, { "key": "deepseek", "role": "model" } ], "themes": [ "fine-tuning-limits", "model-ownership", "ai-factory-economics", "small-model-viability", "agentic-repair", "validator-authority" ], "finding": "The question was cost per task. The finding is that you cannot answer it without first pricing the fine-tune, and the fine-tune is capital that amortizes, not an implementation detail. A week of failing to make an owner’s fine-tune beat a foundation model at code repair produced the thing worth keeping: a decision procedure for when fine-tuning pays, sitting under a compliance override that decides self-hosting first, no matter the capability. Self-hosting has several doors, and fine-tuning is the narrowest: compliance forces it, off-the-shelf open weights can win it with no tune at all, and only then does capability fine-tuning apply, gated by five conditions code repair meets two of. Most enterprise domains that look like fine-tuning problems are retrieval problems wearing the costume.", "video": null, "lab_detail": "https://labs.layer2c.com/downloads/trained-by-whom-lab-detail.md", "question": "We set out to measure cost per gate-verified task, own hardware against hosted API, and answer whether an expensive owned box earns its price. The measurement kept stalling on a cost we had minimized: the cost of the model itself. You do not get the model for free. You build datasets, run a tune, evaluate it, throw it away, and try again, and the infrastructure the discarded models burned does not vanish because the models did. Pricing a task means pricing that search. This lab ran the search to the end on one domain and came back not with a number but with the procedure that decides whether the number is worth computing at all.", "load": "One campaign, six days, two owned DGX Sparks, one rented MI300X, and four hosted APIs. The eval instrument: the 22-repair pool and deterministic gate of Labs 011-013, unchanged. New apparatus: an 82-task disjoint training pool mined from a dozen fresh repos, a 59-trajectory teacher corpus, a 1,719-example synthetic precision factory, four trained adapters (three imitation/skill tunes plus one self-consolidation LoRA that ran), a self-generated harvest of 246 gate-passed loop solutions, roughly 150 base and tuned agentic sessions across a dozen arms, a weights-by-harness read, Gemini and DeepSeek baselines, a trajectory taxonomy, two-Spark DDP training, and a mechanical venue preflight that now refuses bad launches. Metered spend roughly $73; owned hardware roughly 90 GPU-hours; two host reboots; every invalidated arm archived, not deleted.", "verdict": { "scope": "Measured on localized repair with an executable test: one 22-task pool, one open-model family as the tuning subject, one week of checkpoints and prices. The decision procedure is the deliverable; the code-repair verdict is the worked example that produced it. The open edges are a domain that satisfies all five conditions, repository-scale work without localization, the file-geometry wall, and a clean re-measurement of harness edit-landing once the tool-call parser was corrected.", "independence": "Self-funded editorial. Disclosure: Google Cloud is a client of The CTO Advisor LLC, and Google models sit at the center of this lab as both tuning subject (Gemma 4) and benched baselines (Gemini 2.5 and 3.6 Flash, Vertex). Anthropic’s Claude Code is the primary harness and is itself implicated in the findings. Hot Aisle, AMD, NVIDIA, OpenAI, Mistral, Alibaba, and DeepSeek are not clients. No vendor commissioned, funded, or previewed any of this.", "calls": [ { "kind": "do", "text": "settle compliance first, then run the five-condition test. Compliance is an override, not a condition: if the data cannot leave the VPC, or the workload is regulated or air-gapped, you self-host regardless of capability and the conditions below do not apply. Where the hosted frontier is a legal option, the test gates capability fine-tuning: the model has to fail your gate not your budget, the missing capability has to be behavior a corpus cannot inject, a deterministic gate has to exist, the value and volume have to amortize a fixed cost, and you have to price the whole how including the enforcement the tune does not remove. Cost tuning to distill a cheaper model that matches an expensive one is a separate branch that lives in the volume math, not this test." }, { "kind": "dont", "text": "treat the fine-tune as an implementation detail inside a cost-per-task number. It is capital. It amortizes across every task it solves, which makes fine-tuning volume economics: catastrophic over one task, invisible over a million of one shape. Any cost model that expenses the training run and forgets the search that preceded it, and the scrap the search burned, is pricing the wrong thing. This search cost roughly $73 and ninety GPU-hours to buy a two-task gain that a hosted model beats outright." }, { "kind": "dont", "text": "assume owning the weights buys you a corpus’s worth of knowledge. Retrieval injects what a model knows. Fine-tuning changes how it acts. If a corpus plus a frontier model clears your gate, the task was a RAG problem wearing a fine-tune costume, and most enterprise domains that look proprietary are exactly that. Fine-tuning’s defensible territory is behavior a corpus cannot supply, which is a narrow intersection, not a default." }, { "kind": "do", "text": "expect the deterministic gate to survive the tune, and budget it. The fine-tune moves behavior most of the way and a hard invariant still leaks the base model’s prior. This practice’s voice work is the proof: even fine-tuned and prompt-guarded, the model reverts to em-dashes, and only deterministic tooling in the loop enforces the pattern. That gate is a recurring cost that never amortizes. Owning the model does not retire the validator. It shifts work onto it." }, { "kind": "do", "text": "match the apparatus to the model and get the serving contract right before you read a single score. In the loop the consolidation tune reached 14 of 22 against a fair untimed base of 12, a nudge inside noise. On the harness the story was a serving bug: the wrong tool-call parser silently dropped the model’s native emission, and corrected, the untuned base scored 15 where it had shown six to eleven. Loop and harness are different apparatus, so do not read the loop’s 14 against the harness’s 15. Serving correctness is part of the model contract, and a mechanical preflight, envelope, arm-scale prefill, end-to-end tool smoke, provenance stamp, is now the gate every arm passes first." } ] }, "objection": { "q": "You just tuned badly, and your best tune only gained two tasks. Better data or RL closes it.", "a": [ "The best-designed tune did run, and it worked: a self-consolidation LoRA trained on the model’s own gate-passed loop solutions scored 14 of 22 in the loop against a fair untimed loop base of 12, a two-task delta that sits inside sampling noise on a pool this size, zero regressions. That is the best the reachable families produced as we reached them, and it is the point. A nudge over base still loses outright to a mini-class hosted model that solves all 22 for $3.14 in forty minutes. The tune succeeding does not move the domain; it confirms the domain belongs to the foundation model even when the tune helps.", "The families an owner can reach are imitation, isolated-skill, and consolidation, and all three are measured here. What remains is frontier-scale RL post-training, which an owner cannot buy as a process, only as a product. That product sits in the open-weights catalog: Devstral and Qwen3-Coder land their edits where Gemma’s family does not, and DeepSeek V4 Flash swept this pool twice on owned clustered hardware for nothing but memory and hours. Acquiring the capability by model choice costs nothing. Acquiring it by training cost this lab a week to establish that, for this domain, it should not have been attempted, and the enterprise version runs the same shape at a thousand times the bill." ] }, "numbers": [ { "label": "The frontier the tune must beat", "value": "mini 22/22 · $3.14 · 40 min", "note": "gpt-5.4-mini, 10-turn median; the whole pool solved for the price of a coffee, which is the first-condition failure for code repair" }, { "label": "Owner fine-tunes, gate-verified gain", "value": "3 tunes · 0 · then +2 (noise)", "note": "imitation and isolated-skill: zero; self-consolidation LoRA: 14/22 vs a fair untimed loop base of 12/22, a two-task delta inside sampling noise on 22 tasks, and it still loses to the frontier" }, { "label": "Harness wall was a serving bug", "value": "base 6-11 → 15/22", "note": "the wrong tool-call parser silently dropped Gemma’s native emission; the gemma4 parser recovered it; the serving contract is part of the model" }, { "label": "The dispatcher, per verified repair", "value": "loop ~$0 · mini $0.14 · DSV4 $0", "note": "loop clears 17/22 deterministically; mini 22/22; DeepSeek V4 Flash 22/22 twice, 7-min median, on two clustered Sparks" }, { "label": "Fine-tuning is volume economics", "value": "fixed ÷ tasks solved", "note": "catastrophic over one task, invisible over a million of one shape; the term that was invisible while we tested a domain the frontier owns" }, { "label": "The how is layered", "value": "tune + non-amortizing gate", "note": "fine-tune moves behavior most of the way; a hard invariant leaks the base prior (em-dashes) until a deterministic gate enforces it, and that gate is recurring cost" }, { "label": "Build vs buy does not zero the term", "value": "validation + transfer risk", "note": "DeepSeek community tune: free download, real GPU-hours to stand up, and it worked only inside its own training scaffold" }, { "label": "Cost of the search, cheat math", "value": "~$4,100 · one workweek · $105/hr blended", "note": "compress to a single workweek at rented-MI300X speed (≤40 GPU-h vs ~90 on the owned Spark); blended rate $103-106/hour, of which the GPU is $3 and the engineer is ~$100; ~40h × ~$105 ≈ $4,100-4,250. Compute is 3% of the hour; the human is the bill. Metered compute alone was ~$73" } ], "opened": [ { "q": "What domain satisfies all five conditions?", "a": "Code repair fails on the first: a hosted model clears the gate for pocket change. The domain where fine-tuning pays needs the foundation model to fail the gate, the missing capability to be behavior a corpus cannot inject, a deterministic gate to exist, and the volume to amortize the build. The Articul8 sponsored work named the Layer 2C reasoning plane but lacks a workload to test at this task-level economics. Finding one, and building its gate before any GPU spend, is the next lab. The gate is the hard part; no gate, no economics." }, { "q": "How far does the parser correction reach?", "a": "The harness numbers phase one read as a model deficit were measured with a tool-call parser that dropped Gemma’s native emission. Corrected, the untuned base jumps from six to eleven up to 15 of 22. The tool-trained peers were parsed correctly, so the 37%-versus-96% edit-land comparison was partly apples to oranges. A clean re-measurement of edit-landing across every model with the correct parser was not completed before the rented card came down. Until it is, treat the phase-one execution-deficit finding as reopened, not overturned." }, { "q": "Does the dispatcher survive losing localization?", "a": "Every task here arrived with its files named, which switches off the one advantage agentic scaffolding is priced for. Repository-scale fault localization is the harness’s home game and the loop cannot attempt it. Whether triage-then-escalate holds, and where the local rung’s share falls, when the pool withholds file names is the next pool to mine, and the domain hunt above may find its gate there." }, { "q": "Whose harness serves whose weights?", "a": "Google’s Antigravity was probed as the bring-your-own-model path for Gemma and closed as unreachable: the desktop app silently declines a local endpoint, and the CLI, inspected at the binary level, lists hosted models only, carries a Google-and-Vertex provider enum with no OpenAI-compatible entry, and gates custom models behind a server-side experiment flag. Local weights under Google’s harness are not routable by any visible surface. The managed floor has moved up into the harness layer, and a routing check that counts requests at the serving log is the mandatory gate for any bring-your-own-model claim." } ], "not_proved": [ "It did not prove owner fine-tuning never pays. It proved that on localized code repair, with one open-model family as subject, three reachable families bought nothing and the best-designed fourth nudged the count by two tasks, a delta inside sampling noise on 22 tasks, measured on one loop configuration without multi-seed runs or confidence intervals. The decision procedure is the transferable claim; the code verdict is one instance of it. The bounded-generation regime where Lab 002 measured a 3x owner-LoRA gain stands untouched, and is itself an instance where the five conditions plausibly hold.", "The parser correction reopens the phase-one harness findings rather than cleanly replacing them. The untuned base scores 15 of 22 with the correct tool parser, but a full re-measurement of edit-landing across all models on the corrected serving was not finished before the card came down. The execution-deficit story is reopened, not overturned.", "The consolidation tune’s two-task gain is measured in the loop against a fair untimed control. Its harness arm was torn down before completion as immaterial to the conclusion, so the tune’s effect on the tool-driven surface is only partially observed.", "The five-condition procedure is validated against one domain that fails it, not against a domain that passes. Its predictive value on a passing domain, the case where fine-tuning is supposed to win, is asserted from mechanism and awaits the next lab. The procedure also gates capability fine-tuning only; the distillation branch (a cheaper model tuned to match an expensive one at volume) is named but not measured here, and the compliance override that precedes all of it is a constraint this lab asserts, not one it tested.", "The DeepSeek V4 Flash sweep is this pool only, on a two-box cluster this practice already owned, and whether it survives an un-localized pool or a different task class is unmeasured.", "Roughly a quarter of the campaign’s spend was avoidable operator error, including two host reboots from pushing unified memory past its wall. The gates built from those errors are asserted to prevent recurrence from mechanism, not yet measured across a subsequent campaign." ], "executive_summary": [ "The lab set out to price a task and discovered it could not, because the model is not free and the fine-tune that produces it is capital, not an implementation detail. The deliverable is the decision procedure that comes before the price: five conditions that have to hold together before fine-tuning, and therefore self-hosting, is the right lever. The foundation model has to fail your gate rather than your budget. The missing capability has to be behavior a retrieval corpus cannot inject. A deterministic gate has to exist. The value per task has to be high and the volume large enough to amortize a fixed cost. And you have to price the whole how, including the deterministic enforcement the tune does not remove.", "Code repair was the worked example, and it fails the test on the first condition. A mini-class hosted model solves the whole 22-task pool for $3.14. Three owner-side tunes, imitation and isolated-skill, produced no gate-verified improvement, and the imitation tune was actively destructive. The fourth and best-designed tune, a self-consolidation LoRA trained on the model’s own gate-passed loop solutions, did run: 14 of 22 in the loop against a fair untimed loop base of 12, a two-task delta that sits inside sampling noise on a pool this size. A tune that at best nudges the count and still loses outright to the frontier is the cleanest proof that the domain belongs to the foundation model even when ownership works.", "Two corrections sharpened the picture. The harness “mechanical wall” that phase one read as a model deficit was substantially a serving bug: the tool-call parser was dropping Gemma’s native emission, and with the correct parser the untuned base scored 15 of 22 instead of six to eleven. The serving contract is part of the model, and getting it wrong reads as incapacity. And the fine-tune never retired the validator: the practice’s own voice work shows a fine-tuned, prompt-guarded model still reverting to em-dashes until deterministic tooling enforces the pattern. The how is layered, and the bottom layer is a gate that never amortizes.", "The economics generalize as amortization and branch order. A fine-tune is a fixed cost divided across every task it solves, so fine-tuning is volume economics: catastrophic over one task, invisible over a million of one shape. Build or buy does not change the term, only its shape, and the buy path carries a transfer risk the build path does not, measured here when a community tune worked only inside its own scaffold. Serving cost, the branch the original spec opened on, is the last branch, not the first: above it sit yield, apparatus fit, whether the vendor trained the capability, and whether you needed to train at all or could have retrieved. And self-hosting is not the same decision as fine-tuning: compliance forces it above any capability question, and off-the-shelf open weights can win it with no tune at all, which is why the frontier-owns-code verdict is a verdict about fine-tuning, not about owning hardware.", "The campaign priced its own search, which is the exhibit the fine-tuning pitch never includes. Roughly $73 metered, ninety owned GPU-hours, six days, and two host reboots to buy a two-task nudge inside noise, about a quarter of it avoidable operator error now encoded as mechanical gates. Priced the way a business incurs it, compressed to a single workweek at rented-MI300X speed, the resource cost is around $105 an hour, of which the GPU is $3 and the engineer is the other hundred. Discovery is the expensive part, and it is expensive even when it is disciplined. The number a self-hosting business case needs is not the training run. It is the search, and the search is paid in attended hours, not dollars of electricity." ], "dapm": [ { "instrument": "4plus1", "layer": "layer0", "placement": "Retained", "note": "Two altitudes, one placement. The owned boxes carried training, serving, loops, and diagnosis at zero marginal dollars, including a two-Spark DDP run once the fabric proved bandwidth-bound rather than latency-bound. The cluster’s highest use surfaced late: unified memory holds an open frontier-scale model that sweeps this pool. Capacity, not capability, is what ownership buys, and a host reboot twice was the price of pushing memory past its wall." }, { "instrument": "4plus1", "layer": "layer2b", "placement": "Retained", "note": "The deterministic layer is the part that never regressed and the part the whole finding rests on: gate, loop, crash-safe results, a venue preflight that refuses bad launches, and, this campaign, a tool-call parser correction that recovered a model the wrong parser had buried. Every lesson that lived as code survived; every lesson that lived as agent memory was repaid this week. The runbook is the control plane; agent memory is a cache." }, { "instrument": "4plus1", "layer": "layer2c", "placement": "Retained via selection, and priced", "note": "The repair-reasoning seat stays shoppable, and this lab prices it. An owner controls the apparatus and the serving conditions for free and the vendor’s post-training only through model selection. The reachable tune bought two tasks and still lost to the frontier on cost. Retain the seat by choosing tool-trained weights or matching the apparatus to untooled ones, and fine-tune only where the five conditions hold. The determines-done authority never moved." } ], "seam_map": [], "writeup": [ "The lab began as a cost-per-task measurement: run one agentic repair pool against owned hardware and against hosted APIs, and find what a solved task actually costs each way. The measurement kept stalling on a term the original model had minimized. You do not get the model for free. Testing whether an owner’s fine-tune beats a foundation model means building the fine-tune, and building it means datasets, a training run, an evaluation, a disappointing result, a data change, another run, and a pile of discarded checkpoints whose infrastructure cost did not vanish when the checkpoints did. Pricing the task means pricing that search. So the deliverable stopped being a number and became the procedure that decides whether the number is worth computing: five conditions that have to hold together before fine-tuning, and therefore self-hosting, is the right lever.", "Code repair was the worked example, and it fails on the first condition. A mini-class hosted model solved the entire 22-task pool for $3.14 in forty minutes, which means the foundation model fails nobody’s gate here, only a budget the frontier already undercuts. Against that floor the owner tunes ran. Three of them, imitation from a teacher and an isolated-skill precision factory, produced no gate-verified improvement, and the imitation tune was worse than nothing: it transferred the teacher’s brevity without the precision that makes brevity work and damaged behaviors the base model already had. The fourth was the one the arithmetic said to build: a self-consolidation LoRA trained on the model’s own gate-passed loop solutions, harvested from a disjoint pool. It ran. Fourteen of 22 in the loop against a fair untimed loop base of 12. On twenty-two tasks a two-task delta sits inside sampling noise, so call it what it is: a small nudge, not a breakthrough, measured against the loop’s own base and not against the harness. A tune that at best nudges the count and still loses outright to a $3.14 hosted run is the cleanest proof that the domain belongs to the foundation model even when ownership is doing its job.", "Two corrections kept the read honest. The harness failure that phase one had called a model deficit, a 37% edit-land rate against tool-trained peers near 96%, was substantially a serving bug. The tool-call parser was dropping Gemma’s native emission format, scoring correctly-shaped tool calls as plain text. With the correct parser the untuned base scored 15 of 22 where the wrong one had it at six to eleven. That does not erase the peer gap, and a clean cross-model re-measurement was not finished, but it relocates a chunk of the wall from the model to the serving contract, which is itself part of the model. Two apparatus, two numbers: the loop’s 14 and the harness’s 15 are different instruments and do not compare to each other. The second correction is that the tune never retired the validator, and the parser is the in-domain proof of it: a deterministic layer had to decode what the model would not reliably emit before the harness could score it. The same pattern shows in this practice’s voice work, where a fine-tuned, prompt-guarded model still reaches for the em-dashes a gate has to strip. The how is layered. The fine-tune moves behavior most of the way, and a deterministic layer holds the invariant the base prior keeps violating.", "From there the economics generalize, and they generalize as amortization. A fine-tune is a fixed cost divided across every task it solves. Over one task it is catastrophic; over a million tasks of one shape it rounds to zero. Fine-tuning is volume economics, and code repair fails on volume as surely as on the frontier price: even a free-to-run tune has to amortize its build against a domain where the frontier already wins per task. Buying the tune instead of building it does not zero the term, it reshapes it. A community adapter is a free download and a real bill in GPU-hours to stand up, and it carries a transfer risk the build path does not, measured here when a well-regarded community tune worked only inside its own training scaffold and nowhere else. Either way you pay to learn whether the weights clear your gate.", "The capability question sits underneath all of it, and it is the one most enterprises get backward. Retrieval injects what a model knows. Fine-tuning changes how it acts. If a corpus plus a frontier model clears the gate, the task was a retrieval problem wearing a fine-tune costume, and the proprietary-knowledge domains that look like fine-tuning candidates are usually exactly that. Fine-tuning earns its cost only where the missing thing is behavior a corpus cannot supply, the gate rejects the frontier, the volume amortizes the build, and the value justifies the search. That intersection is small, which is the finding, not a disappointment: real fine-tuning use cases are rarer than the market implies. And the arithmetic for self-hosting falls out of the same order, once you stop fusing it with fine-tuning. Self-hosting is forced by compliance regardless of capability, no amount of frontier quality overrides a workload that cannot leave the VPC. It can also be won outright by off-the-shelf open weights that need no tune, the way an open model swept this pool self-hosted and untrained. Fine-tuning is only the narrowest door into it. Below all of that, serving cost is the last branch: above it sit yield, apparatus fit, whether the vendor trained the capability you need, and whether you needed to train at all. Own the model when compliance demands it, when open weights already win, or when the five conditions hold and the volume is yours. Rent the frontier otherwise, which is more often than the pitch admits. And price the search in the currency that dominates it, which is not GPU-hours but attended engineer time and calendar, the terms the owned box hides and the CFO does not." ], "abstraction": [ "The transferable output is a decision procedure, and it starts above the economics with an override. Compliance is not a condition to weigh, it is a veto that decides first: if the data cannot leave the VPC, or the workload is air-gapped, regulated, or contractually bound to stay in-house, you self-host, and no amount of frontier capability changes that. Capability never trumps compliance. Everything below assumes the hosted frontier is a legal option in the first place.", "With that settled, separate two decisions the pitch fuses. Self-hosting has several doors, and fine-tuning is the narrowest. Compliance forces it. Off-the-shelf open weights can already beat the frontier on price for your volume with no training at all, the way an open model swept this bench’s pool self-hosted and untuned. And fine-tuning itself splits into two motives that do not share a rule: capability tuning makes a model do what it could not, and cost tuning, distillation, makes a cheaper model do what an expensive one already does and is justified by volume and price delta alone. The five conditions below gate capability tuning. Distillation is a separate calculation that lives entirely in the volume term.", "For capability tuning, five conditions have to hold together. One, the foundation model fails your gate, not just your budget; if it produces acceptable output at any price, capability is not what is missing, and you are looking at a distillation or a cost problem instead. Two, the missing capability is behavior, not knowledge; retrieval injects what a model knows, fine-tuning changes how it acts, and if a corpus plus a frontier model clears the gate it was a retrieval problem in a fine-tune costume. Three, a deterministic gate exists; no gate, no cost-per-verified-task, no economics, and this is the condition most domains lack. Four, value per task is high and volume is large, because a fine-tune is a fixed cost amortized across every task it solves. Five, you have priced the whole how, including the deterministic enforcement the tune does not remove.", "That fifth condition is the one the market misses. A fine-tune moves behavior most of the way and a hard invariant still leaks the base model’s prior, so a deterministic layer enforces the last mile. This bench saw it twice: a tool-call parser that had to decode the model’s native emission before the harness could score it at all, and, in this practice’s voice work, a gate that strips the em-dashes a fine-tuned model still reaches for. The enforcement is a recurring cost that never amortizes while the fine-tune is capital that does, so more fine-tune converts flow cost into amortizable capital. This is Deterministic Code In The Loop: the model reasons, the deterministic validator enforces, the control point is the gate. Owning the model does not retire the validator; it shifts work onto it.", "Then price the search honestly, because the search is the line item the pitch omits and the one the CFO finds. The cost is not the GPU-hours. It is attended human time at a real rate plus compute at the venue price, and the two are not the same clock: this campaign’s ninety GPU-hours were not ninety hours at a keyboard, because guards, beacons, and overnight runs decoupled the wall time from the labor. Price the attended fraction at a real rate, add compute, then ask the counterfactual the owned box hides: would faster rented hardware have compressed the wall time enough to cut the attended time and the calendar, even at higher compute cost per hour? The cheat math makes it vivid: compress the search to a single workweek at rented-MI300X speed and the blended rate is $103 to $106 an hour, of which three dollars is the GPU and a hundred is the engineer, so compute is three percent of the hour. It is the cheap, last branch, sitting below yield, apparatus fit, what the vendor trained, and whether you needed to train at all. Own the model when compliance demands it, when off-the-shelf weights already win, or when the five conditions hold and the volume is yours. Rent the frontier otherwise, which is more often than the pitch admits." ], "method": [ "Self-funded editorial. The instrument is loopcontrolbench, open under MIT: the reproduce-or-drop gate, the sandboxed runner, both harness drivers, and the additions this campaign contributed, a mechanical venue preflight (envelope, arm-scale prefill, end-to-end tool smoke, provenance stamp), the trajectory taxonomy analyzer, the land-rate gate, the synthetic precision factory, the loop-solution harvester, and a two-Spark DDP training path. The teacher corpus, the trained adapters, and any vendor-assessment consequences stay proprietary. Cross-lab operating discipline accumulated into specs/BENCH-METHOD.md, including the rules this campaign paid for: the tune must earn its place against a clean baseline, serve at the reference envelope with the correct tool-call parser, verify routing by counting requests, cap the loop at two turns, and kill by PID never by pattern.", "Every invalidated arm is archived with its defect named, not deleted: a proxy that routed to the wrong API surface, a context cap below the reference envelope, a ceiling miscalibrated for contention, and a tool-call parser that dropped the model’s native emission. Serving configuration travels with every result via the venue note and a per-task backend probe. The comparisons are kept distinct by design: base versus tuned is the measurement, tuned versus teacher is distillation fidelity, and local versus managed serving is the economics, which this campaign resolved one branch above where the spec opened it, and then generalized into the decision procedure that is the lab’s finding." ], "assessments": [], "publisher": "The Advisor Bench LLC", "program": "Layer2C Labs", "url": "https://labs.layer2c.com/labs/trained-by-whom", "markdown_url": "https://labs.layer2c.com/labs/trained-by-whom.md", "json_url": "https://labs.layer2c.com/labs/trained-by-whom.json" }