# Recovered capacity is real, and it fails honest

> Lab 009 · Editorial lab · Status: published  
> Published by The CTO Advisor LLC · Layer2C Labs

**Question:** Enterprises commonly carry idle owned hardware or unused cloud-spend commitments above their operating baseline: headroom producing nothing between peaks. This lab asked one question of that idle capacity: can a smaller model clear real, verifiable work on it, judged by the same deterministic validator that judged the big model? Across three models, four quantization tiers, and two substrates, the answer came back yes, with the load-bearing detail attached: in 585 scored attempts, the audit found no validator pass that failed the available held-out checks. The work that cleared was verified as far as the instrument can see. The work that failed went to a human queue. No token meter ran.

**Load:** The Lab 3 instrument, unmodified and pin-verified: 39 coding tasks, most reconstructed from real bug fixes in real Python libraries (httpx, requests, dateutil, more-itertools, arrow, click, and others), each judged by layered deterministic tests including hidden references the model never sees. Workers: Gemma 4 12B dense (bf16 through Q3_K_M) and Gemma 4 26B-A4B mixture-of-experts, on the DGX Spark control and on AWS m8i.12xlarge (Intel Xeon 6, Granite Rapids, AMX, SMT off). Total cloud spend: $28.32.

## Executive Summary

The lab collapsed to a single measured question on purpose. Labs 3 and 7 already priced the throughput of commodity Xeon and the behavior of the Loop Control architecture; what nobody had measured was whether a smaller model holds the success rate on the same problems, judged by the same pinned validator. Everything else is arithmetic. The bench filled exactly that cell, then kept filling cells the answers opened.

The controlled ladder ran one variable at a time. Run A changed only the model (12B dense versus the 26B reference, same Spark silicon): quality held at 20 of 39 tasks, and on the honest cross-run metric the 24-task overlap with the Lab 3 baseline reads 13 versus 13 unanimous-pass, dead even; the any-pass gap belongs to the metric caveat below. Run B changed only the substrate (the same 12B moved to a rented Granite Rapids shape): quality survived, gaining one task and losing none. The 26B on the same rented shape tied the 12B at 21 while finishing the identical protocol in 2.24 hours against 5.54. That is a configuration comparison, not a single-variable result: architecture, precision, and runtime moved together, with sparse activation the leading explanation for the gap.

The descent found the floor and bracketed it from both sides. Q8_0 and Q4_K_M held the exact pass set of the bf16 original, tier after tier; Q3_K_M broke by five tasks net, triple-confirmed; and the E4B size probe broke below the boundary from the other axis at 15 of 39. The two breaks have different textures worth knowing: precision broke toward failure-to-fix (more honest fails, little collateral), while size broke destructively (seven regression verdicts, five of them real broken-worse-than-found edits). Same safety property either way: at every tier, including both broken ones, the false-pass audit came back zero. Quality degrades closed on this instrument, all the way down — but what lands in the human queue differs in kind, and a team sizing its own boundary should look at both axes.

The deterministic component of this architecture is the validator, not the model, and the data kept illustrating why that placement is the thesis. The Lab 3 baseline flickered verdicts at temperature zero; the 12B ran verdict-identical for 117 straight traces; different silicon produced different tokens from the same weights. None of it mattered to the output, because reaching the output requires clearing a gate that behaves the same way every time. That is Deterministic Code In The Loop, priced: the worker is fungible, the gate is the asset.

## DAPM Table — Authority Verdict

| Layer | Placement |
| --- | --- |
| layer0 | Retained / Delegated |
| layer2b | Retained |
| layer2c | Retained |

## Detailed Writeup

The instrument is the credibility, so it ships first. The task set is not synthetic toys: most of the 39 tasks are reconstructed from real bug fixes in real Python libraries, each pinned to a repo and commit, presenting the codebase as it stood with the historical bug present. Detection is layered, and each layer catches a different failure: feature tests verify the fix works, regression tests catch collateral damage, hidden reference tests the model never sees catch gaming, and a static gate kills malformed output before any code runs. The validator was pinned bit-identical across every run by manifest hash and by functional census, and the audit that verified it also X-rayed it: one weak test, two tasks without hidden references, six structurally unpassable tasks, all found by running the same audit against the original Lab 3 baseline, all carried like-for-like on both sides of every comparison. A lab that will not X-ray its own instrument has no business grading anyone else’s.

The design was a controlled 2x2 against the Lab 3 reference, one variable per step. Change only the model: the 12B dense held quality on the Spark. Change only the substrate: the same model held quality on rented Granite Rapids, and at temperature zero the different silicon produced different tokens with the same task-level verdicts, which is worth a sentence of its own: determinism is a property of a model-substrate pair, not of a model. Then the additions the data demanded: the 26B mixture-of-experts on the same rented shape tied the 12B’s yield while finishing in less than half the wall time, and the quantization ladder descended until it found the floor. Every cell was scored by the same pinned validator with the same false-pass audit. The one cell the ruling names that began as a composite, the 26B at Q4, was queued for measurement rather than asserted, because the difference between those two words is this program’s entire brand.

The false-pass column is the finding under the finding. A batch system that ships defects quietly is worse than no system, and the audit existed to catch exactly that: every validator pass, re-run against held-out tests. Zero confirmed false passes, everywhere, including at the tier where quality broke. When the Q3 model got worse, it got worse honestly: more failures routed to the queue, verdicts churning in both directions, nothing slipping through as false success. The architecture’s promise, that the failure mode is a person looks at it, held at every point the lab could measure, and the hidden-reference layer is what made that promise falsifiable rather than rhetorical.

The drift this lab caught in its own analysis is worth publishing because everyone will make it. Midway through, the analysis began treating model determinism as a virtue: celebrating zero-flip runs, framing the baseline’s verdict flicker as contamination. That inverts the thesis. In Deterministic Code In The Loop, the deterministic component is the validator; the baseline’s flicker is an argument for the architecture, because a variance-prone worker behind a deterministic gate is safe by construction, and on idle cycles its retries become verified mining. The corrected metric hierarchy ships on this page: per-attempt verified yield is the economic statistic, determinism is an operational property that sets retry policy, and comparing best-of-n numbers against single-run numbers across different variance profiles is a quiet corruption that likely infects more published model comparisons than anyone has checked.

The economics land where the thesis pointed. On the rented shape, a verified bug fix cost about nine cents of compute at on-demand rates and six at the measured Savings Plan rate. On owned idle hardware, the marginal cost approaches incremental power, cooling, and operations; inside an otherwise-unused Savings Plan commitment, the incremental bill can approach zero, subject to commitment coverage. Against that denominator, a 54% yield is not a grade on the model. It is recovered value on spend that produced nothing last quarter, with the remainder routed to humans who were always the fallback, and zero tokens metered along the way. The boundary and workhorse coordinates above are this environment’s answer. The instrument, the descent method, and the audit are the transferable part, and they are exactly what a team runs to find their own.

## Assessments at the Time of the Lab

| Vendor | Layer | Grade | As assessed |
| --- | --- | --- | --- |
| NVIDIA AI Platform | Layer 0 · Compute | NVIDIA Strength — Silicon Authority | May 22, 2026 |
| NVIDIA AI Platform | Layer 2B · Runtime | NVIDIA Authority — Inference + Agent Runtime | May 22, 2026 |
| NVIDIA AI Platform | Layer 2C · Reasoning | Runtime Governance Only — Not a Reasoning Plane | May 22, 2026 |
| Intel AI Infrastructure Portfolio | Layer 0 · Compute | CPU Strong, Accelerator Present-but-Thin, No Fabric | June 26, 2026 |
| Intel AI Infrastructure Portfolio | Layer 2B · Runtime | Open Runtime Tooling, No Managed Service | June 26, 2026 |
| AWS AI Infrastructure | Layer 0 · Compute | Custom Silicon Full Stack | July 12, 2026 |

## Method and Disclosure

Editorial and self-funded: $28.32 of cloud spend, all instances terminated and verified, plus owned hardware (the DGX Spark) for the control runs and the entire quality ladder. Disclosure: Intel is a client of this practice; the lab corresponded with Intel about serving configurations during this program, the exchange is paraphrased where referenced, and no vendor commissioned, funded, or previewed anything. The vendor-blessed serving path was tested because Intel named it, on the record, and the result published regardless of direction.

The instrument is the Lab 3 Loop Control harness, unmodified: 39 tasks (most reconstructed from real bug fixes in httpx, requests, dateutil, more-itertools, arrow, click, and other real libraries, pinned to repo and commit), layered deterministic tests including hidden references, pinned bit-identical across every run by sorted-manifest hash and functional census. Task contents and solutions stay private; the method, source repos, audit design, and every rate, count, and cost ship in the raw detail, including the full chronology with its own mistakes: an instrument-location error in the runbook, a hash recipe that had to be re-anchored, two serving-behavior confounds caught at smoke time, and one cleanup error on a pre-existing cloud volume, disclosed to the account owner the hour it happened.

---
*Layer2C Labs · The CTO Advisor LLC · labs.layer2c.com*