{ "slug": "xeon-rematch", "lab_number": 9, "title": "Recovered capacity is real, and it fails honest", "type": "editorial", "status": "published", "sponsor": null, "author": "Keith Townsend", "date": "2026-07-16", "date_iso": "2026-07-16", "layers": [ "layer0", "layer2b", "layer2c" ], "axes": [ { "instrument": "4plus1", "layers": [ "layer0", "layer2b", "layer2c" ], "has_dapm_table": true } ], "supersedes": [], "superseded_by": null, "corrects": [ { "lab": "loop-control", "what": "The escalation statistics. Nine invalid validator checks had booked escalation events for unsolved cases; corrected, the local model was clearing the solvable work on its own and the frontier tier bought throughput, not correct answers." } ], "corrected_by": [], "vendors": [ { "key": "nvidia", "role": "hw" }, { "key": "intel", "role": "hw" }, { "key": "aws", "role": "cloud" } ], "themes": [ "recovered-capacity", "small-model-viability", "validator-authority" ], "finding": "A smaller model on idle capacity cleared real verifiable work under the same deterministic validator; in 585 scored attempts no validator pass failed the held-out checks.", "question": "Enterprises commonly carry idle owned hardware or unused cloud-spend commitments above their operating baseline: headroom producing nothing between peaks. This lab asked one question of that idle capacity: can a smaller model clear real, verifiable work on it, judged by the same deterministic validator that judged the big model? Across three models, four quantization tiers, and two substrates, the answer came back yes, with the load-bearing detail attached: in 585 scored attempts, the audit found no validator pass that failed the available held-out checks. The work that cleared was verified as far as the instrument can see. The work that failed went to a human queue. No token meter ran.", "load": "The Lab 3 instrument, unmodified and pin-verified: 39 coding tasks, most reconstructed from real bug fixes in real Python libraries (httpx, requests, dateutil, more-itertools, arrow, click, and others), each judged by layered deterministic tests including hidden references the model never sees. Workers: Gemma 4 12B dense (bf16 through Q3_K_M) and Gemma 4 26B-A4B mixture-of-experts, on the DGX Spark control and on AWS m8i.12xlarge (Intel Xeon 6, Granite Rapids, AMX, SMT off). Total cloud spend: $28.32.", "verdict": { "scope": "Scoped to falsifiable batch work: tasks with a deterministic validator, here real-library bug fixes judged by executable tests. The safety claim extends exactly as far as the validator’s detection power; deterministic does not mean complete, and a deterministic gate can reproducibly admit a defect its tests do not detect. The ruling answers the one question; the boundary numbers are this environment’s coordinates, not universal constants. The size and shape of the models and compute another team needs is their sizing exercise, and this page ships the instrument and method to run it. Two adjacent cells were deliberately not run and are named in the bound: the 26B MoE at Q4 and the E2B size rung. The ruling does not need them, and asserting cells you chose not to measure is how labs drift into marketing.", "independence": "Disclosure: Intel is a client of this practice, and the lab corresponded with Intel about serving configurations during the program; the exchange is paraphrased where it appears and no vendor previewed or funded any of it. The bench ran on $28.32 of self-funded cloud spend plus owned hardware. No vendor paid for this answer.", "calls": [ { "kind": "do", "text": "Put committed idle capacity to work on falsifiable batches behind a deterministic validator. Measured yield: 51 to 54% of attempts cleared as verified passes among the configurations above the measured boundary (numerators on the card), with the Lab 3 baseline at 47.8% on its own 29-task coverage. On the rented Granite Rapids shape, the full 117-attempt protocol produced 63 verified bug fixes in 2.24 hours: about nine cents of on-demand compute per verified fix, six cents at the measured one-year Savings Plan rate, and an incremental bill that can approach zero inside an otherwise-unused spend commitment." }, { "kind": "do", "text": "Trust the gate, not the model. Zero confirmed false passes in 585 scored attempts across every configuration measured. Failures routed to the human queue or died at the output gate; no confirmed false success cleared the measured gate, and that claim extends exactly as far as the validator and its held-out evidence detect. The failure mode of this architecture is a person looks at it, not a token meter running." }, { "kind": "do", "text": "Size down with a controlled descent. In this environment the measured model boundary landed at a 7.7GB artifact: Gemma 4 12B at Q4_K_M held the identical pass set as its bf16 original across three quantization tiers, measured on the Spark control. Substrate survival was proven at bf16, so the boundary config on rented Xeon rests on two measured edges rather than a measured cell; that bound is stated, not hidden. One rung lower broke, cleanly and honestly. Your boundary will differ; the screen-then-confirm ladder that finds it takes an afternoon." }, { "kind": "dont", "text": "Do not require the model to be deterministic, and do not read variance as defect. The deterministic component of this architecture is the validator. A variance-prone worker behind a deterministic gate fails closed against the defects the gate is built to detect, and on idle cycles its retries become mining: every additional pass it flickers into is checked against the held-out evidence before it counts." } ] }, "objection": { "q": "A 54% pass rate is failure half the time. Why not use a frontier model or a person?", "a": [ "Because the denominator is free and the failures are honest. The 46% that fails costs idle cycles that were already bought and routes to the human who was the fallback before this system existed. The 54% that clears is verified work the human no longer does. The comparison is not this model versus a better worker; it is this yield versus the zero yield the same committed capacity produced last quarter.", "The bigger worker does not change the answer. The 26B mixture-of-experts, the same class of model that anchored the original Lab 3 baseline, cleared the same 21 of 39 tasks on the identical substrate as the 12B, and the baseline itself sits within a few points on per-attempt yield. Within the overlapping task set and measured configurations, the larger worker bought no material verified-yield advantage; its measured advantage was throughput. Correctness lives in the gate.", "And the frontier alternative changes the failure mode, which is the thing this architecture exists to control. A paid-token escalation path fails by accruing incremental charges. This loop fails by growing a queue; its infrastructure is metered too when rented, but no per-token charge accrues. For batch work on committed headroom, a queue is recoverable in a way an open-ended meter is not, and the zero confirmed-false-pass audit is what makes the queue trustworthy." ] }, "numbers": [ { "label": "Confirmed false passes", "value": "0", "note": "Across 585 scored attempts: three models (12B dense, 26B MoE, the E4B size probe), four quant tiers, two substrates. Every validator pass re-checked against hidden reference tests the model never sees. Fail-closed held everywhere, including at the broken tier" }, { "label": "Per-attempt verified yield", "value": "51-54%", "note": "The primary economic statistic, numerators visible: 12B on Spark 60/117 (51.3%), 12B on Xeon 62/117 (53.0%), 26B-Q8 63/117 (53.8%); Lab 3 baseline 43/90 (47.8%) on its 29-task coverage (57.3% restricted to the 24-task overlap, where the 12B reads 39/72 = 54.2%). Attempts are the unit of cost, verified passes the unit of yield" }, { "label": "Cost per verified bug fix", "value": "$0.09 / $0.06", "note": "63 verified fixes in 2.24h: nine cents at on-demand, six at the measured 1yr Savings Plan rate. On owned idle hardware, marginal cost approaches incremental power and operations; inside an otherwise-unused Savings Plan commitment, the incremental bill can approach zero. Zero paid tokens in the loop, by construction" }, { "label": "The boundary (this environment)", "value": "12B at Q4_K_M, 7.7GB", "note": "Identical pass set to bf16 across three quant tiers. Bracketed from both sides: Q3 breaks below it on precision (net -5, triple-confirmed) and E4B breaks below it on size (15/39). Both breaks fail-closed. The descent method ships; your boundary is an afternoon away" }, { "label": "The workhorse", "value": "26B MoE, 2.5x faster", "note": "Same 21/39 yield as the 12B on identical silicon, 2.24h vs 5.54h for the identical protocol. A configuration result: architecture, precision, and runtime changed together; sparse activation is the leading explanation" }, { "label": "Quality across the 2x2", "value": "held, both axes", "note": "Model size (Run A, Spark control): 20/39 vs baseline. Substrate (Run B, Granite Rapids): 21/39, gained one, lost none. One variable moved per run; the validator pinned bit-identical throughout" }, { "label": "Baseline comparison, both metrics", "value": "17 vs 12 becomes 13 vs 12", "note": "The 26B-on-Xeon versus the Lab 3 baseline, 29-task overlap: any-pass reads 17 vs 12, a scary 5-task deficit; unanimous-pass reads 13 vs 12, one task. Any-pass rewards the baseline’s own verdict flicker (4 flip tasks) against a zero-flip run. Never compare best-of-n to single-run across different variance profiles" }, { "label": "Vendor-blessed config", "value": "4.7x prefill, MoE: crash", "note": "llama.cpp-OpenVINO, Intel’s named path: real prefill gains on dense, 16% slower on bandwidth-bound decode, and a repeatable assert running the MoE. Stock llama.cpp remains the only stack that runs the MoE and engages AMX (hardware counters)" }, { "label": "Reserved economics footnote", "value": "Savings Plans, not RIs", "note": "As measured 2026-07-15 (EC2 API, us-east-1), zero classic Reserved Instance offerings for m8i.12xlarge; the reserved analog is a Savings Plan ($1.68/hr 1yr no-upfront vs $2.54 on-demand). A Savings Plan commits dollars, not capacity — the math prices unused spend commitment and says so" }, { "label": "Lab spend", "value": "$28.32", "note": "All cloud legs, instances terminated and verified. The Spark legs and the entire quality ladder ran on owned hardware at zero marginal cost, which is the thesis performing itself" } ], "opened": [ { "q": "Does the yield survive a second substrate class?", "a": "The economic structure is silicon-independent by construction: committed headroom plus a deterministic validator plus a human backstop applies to TPU, APU, and GPU reservations equally. The yield is a property of the model-substrate pair and must be measured per lane; labs 7 and 8 already showed the same model behaving differently across lanes. Lab 10 runs this instrument on a second accelerated substrate, and that is where the generalization is earned or retired." }, { "q": "Does the profile hold on heterogeneous work, and what does variance mining actually yield?", "a": "This lab ran one task domain with an executable validator, the domain where Deterministic Code In The Loop is strongest. Real batches are mixed, and validators weaker than pytest make the false-pass column harder to trust. Separately, the corrected framing implies an unmeasured strategy: a variance-prone worker retried on idle cycles mines additional verified passes. The yield curve of that mining, and where it beats simply running a bigger worker once, is an open measurement." } ], "not_proved": [ "It did not prove the boundary generalizes. Q4 as the floor and Q3 as the break are this environment’s coordinates: this model family, this task set, this validator. The page ships the method to find yours, not the claim that yours matches.", "The headline boundary config on the rented substrate rests on two measured edges, not a measured cell: substrate survival was proven at bf16 and the Q4 floor was proven on the Spark. Two adjacent cells were deliberately not run: the 26B MoE at Q4 (the workhorse claim stands on its measured Q8 cell) and the E2B size rung (E4B already breaks below the boundary). The E4B break itself is a single-screen declaration: its confirmation runs were canceled in the same close-out, with the mitigating fact that every lab 9 tier granted multiple runs was verdict-deterministic. All of it stated, none of it asserted beyond its evidence.", "It did not test heterogeneous batches or non-executable validators. One domain, chosen because its validator is deterministic; the architecture’s scope limit is the validator’s scope limit, exactly as Lab 3 ruled.", "It did not measure the market. Whether reserved Xeon floors are forming at hyperscalers, and why, are hypotheses the lab’s own capacity census declined to confirm; they stay off this page.", "The instrument has known edges, found by this lab’s own audit and disclosed above; three tasks carry no usable held-out evidence and six are structurally unpassable on all sides. Repair is handed to lab 10." ], "executive_summary": [ "The lab collapsed to a single measured question on purpose. Labs 3 and 7 already priced the throughput of commodity Xeon and the behavior of the Loop Control architecture; what nobody had measured was whether a smaller model holds the success rate on the same problems, judged by the same pinned validator. Everything else is arithmetic. The bench filled exactly that cell, then kept filling cells the answers opened.", "The controlled ladder ran one variable at a time. Run A changed only the model (12B dense versus the 26B reference, same Spark silicon): quality held at 20 of 39 tasks, and on the honest cross-run metric the 24-task overlap with the Lab 3 baseline reads 13 versus 13 unanimous-pass, dead even; the any-pass gap belongs to the metric caveat below. Run B changed only the substrate (the same 12B moved to a rented Granite Rapids shape): quality survived, gaining one task and losing none. The 26B on the same rented shape tied the 12B at 21 while finishing the identical protocol in 2.24 hours against 5.54. That is a configuration comparison, not a single-variable result: architecture, precision, and runtime moved together, with sparse activation the leading explanation for the gap.", "The descent found the floor and bracketed it from both sides. Q8_0 and Q4_K_M held the exact pass set of the bf16 original, tier after tier; Q3_K_M broke by five tasks net, triple-confirmed; and the E4B size probe broke below the boundary from the other axis at 15 of 39. The two breaks have different textures worth knowing: precision broke toward failure-to-fix (more honest fails, little collateral), while size broke destructively (seven regression verdicts, five of them real broken-worse-than-found edits). Same safety property either way: at every tier, including both broken ones, the false-pass audit came back zero. Quality degrades closed on this instrument, all the way down — but what lands in the human queue differs in kind, and a team sizing its own boundary should look at both axes.", "The deterministic component of this architecture is the validator, not the model, and the data kept illustrating why that placement is the thesis. The Lab 3 baseline flickered verdicts at temperature zero; the 12B ran verdict-identical for 117 straight traces; different silicon produced different tokens from the same weights. None of it mattered to the output, because reaching the output requires clearing a gate that behaves the same way every time. That is Deterministic Code In The Loop, priced: the worker is fungible, the gate is the asset." ], "dapm": [ { "instrument": "4plus1", "layer": "layer0", "placement": "Retained / Delegated", "note": "The economic object is headroom you already carry, in two distinct forms the page keeps separate: owned idle hardware (the Spark) and unused cloud-spend commitment (the Savings Plan analog; as measured, no classic RI offerings existed for the shape — a Savings Plan commits dollars, not capacity). The lab prices that headroom, not a new purchase. Availability footnote: four AWS launches, zero quota friction, 16 to 29 seconds to SSH." }, { "instrument": "4plus1", "layer": "layer2b", "placement": "Retained", "note": "The runtime is open source (vLLM, llama.cpp) and the control point is the validator and escalation policy, per the Loop Control ruling. The vendor-blessed OpenVINO backend was measured for scope: 4.7x on prefill, slower on bandwidth-bound decode, and unable to execute the mixture-of-experts graph at all. Stock stacks carried every measured cell." }, { "instrument": "4plus1", "layer": "layer2c", "placement": "Retained", "note": "Open weights, small and quantized: the boundary worker is a 7.7GB artifact. Nothing in the loop depends on a garden, an API, or a token meter, and the measured cost of that independence on this workload was a few points of yield against the strongest baseline reading." } ], "seam_map": null, "writeup": [ "The instrument is the credibility, so it ships first. The task set is not synthetic toys: most of the 39 tasks are reconstructed from real bug fixes in real Python libraries, each pinned to a repo and commit, presenting the codebase as it stood with the historical bug present. Detection is layered, and each layer catches a different failure: feature tests verify the fix works, regression tests catch collateral damage, hidden reference tests the model never sees catch gaming, and a static gate kills malformed output before any code runs. The validator was pinned bit-identical across every run by manifest hash and by functional census, and the audit that verified it also X-rayed it: one weak test, two tasks without hidden references, six structurally unpassable tasks, all found by running the same audit against the original Lab 3 baseline, all carried like-for-like on both sides of every comparison. A lab that will not X-ray its own instrument has no business grading anyone else’s.", "The design was a controlled 2x2 against the Lab 3 reference, one variable per step. Change only the model: the 12B dense held quality on the Spark. Change only the substrate: the same model held quality on rented Granite Rapids, and at temperature zero the different silicon produced different tokens with the same task-level verdicts, which is worth a sentence of its own: determinism is a property of a model-substrate pair, not of a model. Then the additions the data demanded: the 26B mixture-of-experts on the same rented shape tied the 12B’s yield while finishing in less than half the wall time, and the quantization ladder descended until it found the floor. Every cell was scored by the same pinned validator with the same false-pass audit. The one cell the ruling names that began as a composite, the 26B at Q4, was queued for measurement rather than asserted, because the difference between those two words is this program’s entire brand.", "The false-pass column is the finding under the finding. A batch system that ships defects quietly is worse than no system, and the audit existed to catch exactly that: every validator pass, re-run against held-out tests. Zero confirmed false passes, everywhere, including at the tier where quality broke. When the Q3 model got worse, it got worse honestly: more failures routed to the queue, verdicts churning in both directions, nothing slipping through as false success. The architecture’s promise, that the failure mode is a person looks at it, held at every point the lab could measure, and the hidden-reference layer is what made that promise falsifiable rather than rhetorical.", "The drift this lab caught in its own analysis is worth publishing because everyone will make it. Midway through, the analysis began treating model determinism as a virtue: celebrating zero-flip runs, framing the baseline’s verdict flicker as contamination. That inverts the thesis. In Deterministic Code In The Loop, the deterministic component is the validator; the baseline’s flicker is an argument for the architecture, because a variance-prone worker behind a deterministic gate is safe by construction, and on idle cycles its retries become verified mining. The corrected metric hierarchy ships on this page: per-attempt verified yield is the economic statistic, determinism is an operational property that sets retry policy, and comparing best-of-n numbers against single-run numbers across different variance profiles is a quiet corruption that likely infects more published model comparisons than anyone has checked.", "The economics land where the thesis pointed. On the rented shape, a verified bug fix cost about nine cents of compute at on-demand rates and six at the measured Savings Plan rate. On owned idle hardware, the marginal cost approaches incremental power, cooling, and operations; inside an otherwise-unused Savings Plan commitment, the incremental bill can approach zero, subject to commitment coverage. Against that denominator, a 54% yield is not a grade on the model. It is recovered value on spend that produced nothing last quarter, with the remainder routed to humans who were always the fallback, and zero tokens metered along the way. The boundary and workhorse coordinates above are this environment’s answer. The instrument, the descent method, and the audit are the transferable part, and they are exactly what a team runs to find their own." ], "abstraction": [], "method": [ "Editorial and self-funded: $28.32 of cloud spend, all instances terminated and verified, plus owned hardware (the DGX Spark) for the control runs and the entire quality ladder. Disclosure: Intel is a client of this practice; the lab corresponded with Intel about serving configurations during this program, the exchange is paraphrased where referenced, and no vendor commissioned, funded, or previewed anything. The vendor-blessed serving path was tested because Intel named it, on the record, and the result published regardless of direction.", "The instrument is the Lab 3 Loop Control harness, unmodified: 39 tasks (most reconstructed from real bug fixes in httpx, requests, dateutil, more-itertools, arrow, click, and other real libraries, pinned to repo and commit), layered deterministic tests including hidden references, pinned bit-identical across every run by sorted-manifest hash and functional census. Task contents and solutions stay private; the method, source repos, audit design, and every rate, count, and cost ship in the raw detail, including the full chronology with its own mistakes: an instrument-location error in the runbook, a hash recipe that had to be re-anchored, two serving-behavior confounds caught at smoke time, and one cleanup error on a pre-existing cloud volume, disclosed to the account owner the hour it happened." ], "assessments": [ { "vendor": "NVIDIA AI Platform", "layer": "Layer 0 · Compute", "grade": "NVIDIA Strength — Silicon Authority", "assessed_date": "May 22, 2026", "version": "v1.3 - Prose-Coherence (GA-Gate)", "source_url": "https://layer2c.com/assessment/nvidia", "snapshot_date": "2026-07-18" }, { "vendor": "NVIDIA AI Platform", "layer": "Layer 2B · Runtime", "grade": "NVIDIA Authority — Inference + Agent Runtime", "assessed_date": "May 22, 2026", "version": "v1.3 - Prose-Coherence (GA-Gate)", "source_url": "https://layer2c.com/assessment/nvidia", "snapshot_date": "2026-07-18" }, { "vendor": "NVIDIA AI Platform", "layer": "Layer 2C · Reasoning", "grade": "Runtime Governance Only — Not a Reasoning Plane", "assessed_date": "May 22, 2026", "version": "v1.3 - Prose-Coherence (GA-Gate)", "source_url": "https://layer2c.com/assessment/nvidia", "snapshot_date": "2026-07-18" }, { "vendor": "Intel AI Infrastructure Portfolio", "layer": "Layer 0 · Compute", "grade": "CPU Strong, Accelerator Present-but-Thin, No Fabric", "assessed_date": "June 26, 2026", "version": "v1.0 — Initial Assessment", "source_url": "https://layer2c.com/assessment/intel", "snapshot_date": "2026-07-18" }, { "vendor": "Intel AI Infrastructure Portfolio", "layer": "Layer 2B · Runtime", "grade": "Open Runtime Tooling, No Managed Service", "assessed_date": "June 26, 2026", "version": "v1.0 — Initial Assessment", "source_url": "https://layer2c.com/assessment/intel", "snapshot_date": "2026-07-18" }, { "vendor": "AWS AI Infrastructure", "layer": "Layer 0 · Compute", "grade": "Custom Silicon Full Stack", "assessed_date": "July 12, 2026", "version": "v1.4 - Single-Vendor API Re-Lens & Label Reconciliation", "source_url": "https://layer2c.com/assessment/aws", "snapshot_date": "2026-07-18" } ], "publisher": "The CTO Advisor LLC", "program": "Layer2C Labs", "url": "https://labs.layer2c.com/labs/xeon-rematch", "markdown_url": "https://labs.layer2c.com/labs/xeon-rematch.md", "json_url": "https://labs.layer2c.com/labs/xeon-rematch.json" }