# They can all write it. Not all of them take direction.

> Lab 016 · Editorial lab · Status: published  
> Published by The Advisor Bench LLC · Layer2C Labs

**Question:** The pitch is that picking the right model tier is the decision that determines whether delegated coding work succeeds. This lab walked a model ladder down from an 80B coder to a 2B, on one from-scratch task judged by a deterministic conformance gate, expecting to find the size where capability breaks. It did not find that. Above roughly 12B every rung wrote a structurally complete, plausibly organized server, and the same 26B model on the same task produced three different approaches in three runs, scoring 7 of 7, then 6 of 7, then 7 of 7, in 12, 79 and 21 minutes. The approach predicted the outcome. The size mostly did not.

**Load:** One from-scratch implementation task: a six-operation OpenAPI contract over a 438-chunk, 258-document corpus fixture, with no reference implementation in the workspace, judged by Schemathesis with all checks on (examples, coverage, fuzzing, and a stateful sequence suite). Scoring is per operation, seven units. Workers: gpt-5.4-mini, Qwen3-Coder-Next 80B-A3B, DeepSeek V4 Flash, and the Gemma 4 family at 26B-A4B, 12B QAT, E4B and E2B, all local rungs served on an owned NVIDIA DGX Spark and driven through the same agentic harness.

## Executive Summary

The lab was staged to find the size where capability breaks and it did not find one. Above the 2B rung every model wrote a structurally complete, plausibly organised server: sensible naming, error handling, comments, an in-memory index built from the fixture. Whether that index was built correctly is where the rungs separate, and at the bottom it was not. The bottom is real and it is low. Gemma 4 E2B drove the agentic harness to a clean exit in three minutes and produced a 108-byte lockfile and nothing else. Everything above it produced a program.

What separated the arms was the approach each session happened to take. The same 26B model on the same task, same weights, same box, same contract, chose surgical edits on one run, whole-file replacement through an exact-match tool on another, and shell heredocs on a third. It scored 7 of 7, 6 of 7, and 7 of 7 in 12, 79, and 21 minutes. Across every arm measured, each one that attempted large exact-match edits lost an operation and each one that did not scored 7 of 7. There is no counterexample in the set.

The mechanism is legible in the trace. The failing run answered a 6,397-character match failure by trying 19,023 characters, then 19,252: it responded to failure by tripling the size of the string it could not reproduce. It then found the working approach at 53 characters, abandoned it, returned to roughly 6,100 and repeated that seven more times. The passing run failed once at 1,648 characters and its next edit was 282. One noticed the strategy was failing. The other did not.

The harness mattered more than the model. The same 12B reached the same 6 of 7 in 63 minutes through the agentic harness and in 4.1 minutes through a constrained loop that asks for whole files and does its own file handling. Fifteen times the wall clock, and roughly 35 times the tokens, for an identical score. Agentic scaffolding is priced for tasks whose inputs must be discovered, and this task hands its inputs over at the start.

The repair loop is worth exactly one turn. With feedback carrying only a failure count, four turns produced 1, 1, 1, 1. With feedback carrying the gate's own remediation text, the same model on the same task produced 1, 6, 3, 1: five operations closed in a single turn, then an overcorrection that flipped a check to its mirror image, then a rewrite that inverted a working handler's semantics. Iteration count did nothing. Feedback content did everything, once.

## DAPM Table — Authority Verdict

| Layer | Placement |
| --- | --- |
| layer2b | Retained |
| layer2c | Retained / Delegated |

## Detailed Writeup

The instrument is a from-scratch task, chosen because translation tasks hide the interesting failure. The workspace receives an OpenAPI contract, a task document and a 438-chunk corpus fixture, and nothing else; the reference implementation exists but is never mounted. Scoring is per operation across six endpoints plus a stateful sequence suite, judged by Schemathesis with all checks enabled. The task document states explicitly that every check the runner performs is part of the bar, including checks the contract does not spell out, and names HTTP method handling as an example. That sentence is load-bearing: it is what makes an unstated requirement a fair test rather than a trap.

The descent found no capability cliff, which was the expected result and did not arrive. gpt-5.4-mini, Qwen3-Coder-Next 80B-A3B and DeepSeek V4 Flash cleared the contract. Gemma 4 26B-A4B cleared it twice in three attempts. The 12B missed by one route in a 405 handler while passing 333 of 333 stateful scenarios. The E4B wrote the longest implementation of any arm and scored 1 of 7 on three small independent defects. Only the E2B failed to produce a program at all. From 2B to 80B, the thing that changed was not whether the model could write the program.

What changed was the density of small precision errors and, separately, whether the model noticed one. Those are two different axes and the lab was measuring neither at the outset. The precision axis is visible in the E4B's escaping error and in the 12B's missing route. The judgment axis is visible only in the trace of a failing run: an exact-match edit tool requires reproducing a target string character-perfect from memory, and one 26B session answered three consecutive failures by enlarging the target from 6,397 to 19,023 to 19,252 characters. It later found the working approach at 53 characters and abandoned it. The passing session, given the same tool and the same task, shrank its target after its single failure.

That is the migration playbook thesis under laboratory conditions. Give three engineers the same playbook and you get three approaches; here one model given one task document produced three. The approaches were not equally good, and the difference was invisible in the playbook's own vocabulary, which talks about contracts and status codes while the failure lived in how a file got edited. No amount of additional playbook detail reaches that axis. Which leaves the acceptance criteria as the only thing that can be held fixed across executors, and that is the argument for a deterministic gate arriving from the delegation side rather than the determinism side. It holds for human executors too.

The human analogy breaks in one place, and the break is the case for the gate. Three engineers taking three approaches can each say why, and each notices when theirs stops working. None of them answers a failed edit by tripling its size, twice, then discards the approach that worked. With human executors a validator catches the occasional bad approach. With delegated model work it is the only thing standing between a thrash spiral and a shipped 6 of 7 that passes 333 of 333 stateful scenarios and reads as finished.

Five control mechanisms came out of this, each forced by an observed failure rather than designed in advance. The gate catches a wrong result. An action-space constraint, removing the exact-match edit tool, makes thrash structurally impossible. A ratchet stops a regressing turn from overwriting a better artifact. An iteration cap stops unbounded self-directed retry. A thrash detector, N consecutive same-mode failures with no reduction in target size, is the correct trigger for escalation rather than iteration. The constraint has a cost the bench measured: removing the edit tool forces whole-file regeneration, and regeneration is what destroyed the repair loop's best result. The mechanisms are complements, not alternatives.

## Method and Disclosure

Editorial and self-funded. The local rungs ran on an owned NVIDIA DGX Spark at zero marginal cost; the metered arms cost a few dollars of API tokens, itemised on the card. Disclosure: Google Cloud is a client of this practice and the Gemma 4 family is the centre of this lab, chosen because it is the only family cached locally at five sizes spanning dense and sparse, which holds tokenizer and training recipe roughly constant while parameter count moves. Anthropic's Claude Code is the agentic harness for every arm and is itself implicated in the findings, particularly the exact-match edit tool. No vendor commissioned, funded, or previewed any of this, and no result was held because of a relationship.

The instrument: a six-operation OpenAPI contract over a 438-chunk, 258-document fixture, judged by Schemathesis 4.24.3 with all checks on, scored per operation. Local rungs served on vLLM 0.25.1 at each model's native context with a per-model tool-call parser, gated before every arm by a tool-call smoke test. The contract, the fixture manifest, the scoring script and the full defect chronology ship in the raw detail; the corpus content stays private.

Twelve harness defects were found and fixed during the program and all of them are disclosed, because a descent is a venue audit that happens to produce capability data. In order: a 32,000-token output reservation that exceeded a small model's context before it saw the task; a context window served below native; an output budget consumed entirely by a reasoning model's thinking; an arm timeout sized against a fast rung; an output contract that required naming a JSON file with a comment, which npm cannot parse; a feedback window that truncated away the validator's remediation text; a driver reading a scoreboard instead of the detailed gate output; a 2,000-character context truncation that hid the worker's own working implementation from it; a hardcoded tool-call parser that fails silently with clean HTTP 200s; the same output reservation, correct for the small rungs, failing a large one mid-write; a controller returning an empty string silently; and a loop that scored a stale artifact three times because it never checked whether a turn wrote a file. Every one produced a confident wrong reading and every one pointed at the model.

---
*Layer2C Labs · The Advisor Bench LLC · labs.layer2c.com*