# The floor is trained, not sized

> Lab 013 · Editorial lab · Status: published  
> Published by The Advisor Bench LLC · Layer2C Labs

**Question:** Lab 012 bracketed the instrument-literacy bar without locating it. Gemma 4 31B failed it, a hosted mini cleared it, and everything in between was guesswork. If you already own the hardware, that gap is the whole decision. So I ran the descent: the same 22 certified bug-fix repairs, the same deterministic test gate, five open-weight models on one NVIDIA DGX Spark, each inside the same headless Claude Code harness. Which of them clears the bar Gemma missed, and what does self-hosting actually cost once you stop counting dollars?

**Load:** The 22 repairs Lab 011's free local loop first missed, each a real bug-fix pull request shipping its own test suite as the pass/fail gate. Six open-weight models served on one 128GB Spark through vLLM: Qwen3-Coder-Next in NVFP4, Devstral Small 2 24B, a Qwen general build, Gemma 4 26B in NVFP4, GLM-4.7-Flash in BF16, and GLM-4.5-Air in AWQ 4-bit. Eighteen arms across two apparatuses, three of them replicated, plus a five-task residual set run twice more to test whether the sharp end was measuring anything at all. Zero metered spend on the local arms. The cost is roughly 107 hours of owned GPU time and a power bill nobody metered.

## Executive Summary

The question was where the instrument-literacy bar sits between a local model that fails it and a hosted one that clears it. It sits lower than the bracket suggested, and it isn't a size. Qwen3-Coder-Next, a 30B-class model quantized to NVFP4 on a single Spark, cleared 18 of 22 gate-verified repairs. Gemma 4 31B, a larger general model, cleared 11.

What separates them is training for tool use, and the evidence is an inversion. In the constrained edit loop Gemma scores 17 and GLM-4.7-Flash scores 4. Swap both into the agentic harness and the order reverses: Gemma drops to 11, Flash climbs to 13. Neither model is better. They're literate in different instruments, and a table that ranks them once is wrong half the time.

Self-hosting cleared the bar and still lost the economics, in a currency the rate card doesn't print. The best local arm spent 11.4 hours of owned GPU time to finish 18 repairs. A hosted mini finished all 22 in 40 minutes for $3.14. That's roughly 18x the wall clock to land four repairs short. Idle hardware makes the dollars free. It does not make the hours free.

One methodological result outranks any model ranking here. The five hardest tasks, the ones the bar was written against, are measuring noise per task. Across five observations one solved every time, one solved once, and the middle three landed near half. The same model took 3 of 5 twice on almost entirely different tasks. Aggregate counts replicate. Individual task verdicts don't, and a bar written as "solved task X" is reading a coin flip.

## DAPM Table — Authority Verdict

| Layer | Placement |
| --- | --- |
| layer0 | Retained |
| layer2b | Retained |
| layer2c | Delegated |

## Seam Map — Readiness

| Function | Readiness |
| --- | --- |

## Detailed Writeup

Lab 012 left a gap nobody could spend against. It showed a free local model failing the instrument-literacy bar and a hosted mini clearing it, which brackets the bar without locating it. For anyone who already owns hardware, that gap is the entire decision. So this lab ran the descent: five open-weight models, one Spark, the same 22 repairs and the same deterministic gate, each inside the same headless Claude Code harness.

The floor is a coder model, and it sits lower than the bracket implied. Qwen3-Coder-Next in NVFP4 cleared 18 of 22. Devstral Small 2 and a Qwen general build cleared 15 each. Gemma 4 26B cleared 14, GLM-4.7-Flash cleared 13. Against Gemma 4 31B's 11, four of five models beat the bar, and the one that beat it hardest was trained specifically for code and tools.

Then the result that should change how anyone reads a model table. Gemma 4 31B scores 17 of 22 in the constrained edit loop, where the model gets one shot to reproduce an anchor from memory and no tools at all. GLM-4.7-Flash scores 4 in that same loop. Swap both into the agentic harness and it reverses: Gemma falls to 11, Flash rises to 13. Neither is the better model. Gemma has byte-exact recall and can't drive tools. Flash drives tools and can't reproduce anchors. Any ranking that puts one above the other is right in one apparatus and wrong in the other.

So what does self-hosting cost? Not money. Every local arm here ran at zero metered spend, which is the whole appeal. The cost lands in the column nobody prices. The best local arm spent 11.4 hours of GPU time to finish 18 repairs, at a 22-minute median and a 108-minute worst case. The hosted mini finished all 22 in 40 minutes for $3.14. That's about 18x the wall clock to end up four repairs short, and the spread holds across the whole field: local medians ran 22 to 62 minutes where every hosted arm ran 1 to 2.

That gap is the product you buy when you pay, and it deserves naming because "the hardware is already paid for" hides it. Idle hardware makes the dollars free. It doesn't make the hours free. If the work is unattended and runs overnight, hours are cheap and self-hosting wins outright. If somebody is waiting on the result, $3.14 buys back most of a working day.

One model looked disqualified and wasn't, which is the most instructive row here. GLM-4.5-Air, 106B parameters in AWQ 4-bit, was staged and appeared not to fit: an agentic harness needs 131K tokens of serving context, the key-value cache for that lives in the same unified memory as the weights, and at the utilization first tried there was no room left for sessions. That reading survived until the serving config was corrected. At util 0.80 the same weights on the same box yield 36.56 GiB of cache, 1.59 concurrent full-context sessions, and 12 of 22 solved. The box was never the constraint. A default `--max-num-seqs` was, the same setting that cost another model three times its cache. Venue qualification is still the right discipline, and it is only as trustworthy as the config it is computed against.

Air also lands where the headline predicts. At 106B it is three times the size of the model that beats it and finishes sixth of seven, one repair above a 31B and six below a 30B coder build. What it does own is speed: a 9.2-minute median against Qwen3-Coder's 22, and 4.5 hours for the full pool against 11.4, because 12B active parameters decode far faster than a dense 24B. Fast, and mediocre. Size bought throughput and no capability.

The last finding is about the bench rather than the models, and it invalidates a claim this campaign already published. The bar was written partly as "clears the residual tasks," the five hardest repairs in the pool. Replicating those five showed per-task outcomes close to random. One task solved in all five observations. One solved in one. The middle three sat near half. Qwen scored 3 of 5 twice on almost disjoint task sets, overlapping only on the task everything solves. The aggregate count carries signal. The task identity does not, so any claim of the form "this model solved that task" is reporting a coin flip somebody happened to watch.

Scope it. One box, one pool, one week of checkpoints, localized repair with an executable test. What carries off this bench is narrower than the numbers suggest: pick for training rather than size, name the apparatus before ranking anything, and price self-hosting in hours, because the dollars will tell you it's free.

## How It Abstracts

Capability in an agentic stack is a pair, not a number. A model carries some literacy in each apparatus you might run it in, and those literacies are separably trained. Ranking models without naming the apparatus produces an ordering that inverts the moment somebody changes the runtime, which an architect can't see from a leaderboard and will feel in production.

Venue qualification precedes capability measurement, and it's computable in advance. For a one-box agentic deployment the arithmetic is weights, plus concurrent sessions times key-value cache per session, plus host headroom, against the box. The trap is that the same arithmetic run against a default serving config retires models that work: it disqualified a 106B model here that later cleared 12 of 22 on the identical hardware. Compute it against a tuned config, and read the cache figure off the server rather than deriving it.

Price self-hosting in the currency it actually costs. Dollars are zero on idle hardware and hours are not, and a 30x wall-clock multiple against a hosted tier decides whether free is worth it. That's AI Factory Economics with a different denominator: the same cost-per-unit-of-output discipline, run against time instead of spend, because on owned capacity time is the meter still running.

## Assessments at the Time of the Lab

| Vendor | Layer | Grade | As assessed |
| --- | --- | --- | --- |
| NVIDIA AI Platform | Layer 0 · Compute | NVIDIA Strength — Silicon Authority | July 31, 2026 |

## Method and Disclosure

Self-funded editorial. The harness is loopcontrolbench, published open under the MIT license at github.com/kltownsend/loopcontrolbench: the reproduce-or-drop gate, the sandboxed runner, and the arm definitions. This lab added a second agentic harness behind an environment switch, so the driver process can change while the prompt, workspace, gate and result schema stay identical, plus a trajectory reader that reports what a session actually did rather than only whether it passed. The vendor assessments and the layer placements this evidence feeds stay proprietary.

Every local arm ran on one NVIDIA DGX Spark with 128GB of unified memory, served through vLLM. Serving configuration is recorded with each result: the checkpoint path, the utilization setting, and the key-value cache the server reported at startup. Earlier arms in this campaign recorded only a proxy alias, which is why one arm in the record cannot name the weights that produced it. Closing that gap is what the provenance capture was added for.

---
*Layer2C Labs · The Advisor Bench LLC · labs.layer2c.com*