# Put the judgment in the constraints, not the weights

> Lab 006 · Editorial lab · Status: published  
> Published by The CTO Advisor LLC · Layer2C Labs

**Question:** The pitch is everywhere: fine-tune a local model on a decade of your published judgment and it becomes you. Lab two ruled "own the weights" and built the kill-criterion that goes with it: if base plus retrieval clears the bar, do not fine-tune. This lab is that criterion firing. Two measured training rounds on an owned DGX Spark lost to their own base model, and both lost to a single paragraph of written positions in the system prompt. Throughout, the box means the compute, serving, and training kept below the platform’s abstraction, and the NVIDIA DGX Spark is that box.

**Load:** A 563-pair instruction dataset built from a 2,727-document advisory corpus and 25 published vendor assessments, trained as LoRA adapters on Gemma 4 26B-A4B and Qwen3-4B, and measured by a 70-probe judgment gate built before the first training run: held-out vendor assessments as answer keys, honesty probes that invite fabrication, framework-discipline probes, and head-to-head comparisons against the author’s real published answers.

## Executive Summary

The lab set out to fine-tune an advisor’s judgment into a local model and ended up measuring where judgment actually lives. The instrument came first: a 70-probe gate with held-out vendor assessments as answer keys, built before training because the June finding was that automated metrics pass models a human reader rejects. Everything that followed was scored by the same gate, the same judge, and a human blind read.

Two training rounds produced one genuine transfer and two persistent losses. Refusal data measurably taught the boundary. Nothing taught the judgment: the tuned models never beat their own base on assessment agreement, lost the advisory head-to-heads 12-1 and 13-0, and the smaller model collapsed into verbatim loops in both rounds. The control ended it. One paragraph of standing positions in the system prompt beat every fine-tuned artifact on every family, then transferred unchanged to the production model on another cloud, where it eliminated live fabrication failures at zero measured cost.

## DAPM Table — Authority Verdict

| Layer | Placement |
| --- | --- |
| layer2b | Delegated |
| layer2c | Retained |
| layer3 | Retained |

## Detailed Writeup

The instrument is the lab. Before any training run, the gate froze 46 assessment probes whose answer keys are the author’s published vendor rulings, held out of training entirely, plus honesty probes engineered to invite fabrication and discipline probes that tempt a model to flatten the 4+1 framework into generic layers. A mechanical repetition screen sits in front of the judges, and 13 advisory questions are scored head-to-head against the author’s real published answers. Every candidate in this lab, local or cloud, tuned or stock, faced the same 70 probes.

The dataset respected one discipline: evidence in the prompt, judgment in the completion, so the model learns the grading move rather than memorizing grades. Construction had its own validator. A first pass of generated question-answer pairs failed the author’s read 8 of 10; the fix was a judge in the construction loop verifying that each answer actually answers its question, which kept 247 of 1,376 eligible documents. The final set was 563 pairs across advisory, assessment, framework catechism, and refusal families.

Round one looked perfect from the inside and failed everywhere the gate looked. Loss curves converged. The artifacts fabricated: asked for a latency comparison nobody benched, the tuned 4B answered with an invented winner, and asked cold for the layer framework it was trained on, the tuned 26B rebuilt a generic compute-network-app stack instead. The 4B also collapsed into verbatim repetition loops on open-ended questions, a failure mode invisible to averaged metrics and obvious to any reader. The diagnosis was the data shape: every training answer rendered confident judgment, so confidence generalized and the boundary did not.

Round two fixed what data can fix and proved the point by contrast. Gentler hyperparameters ended the collapse on the 26B. Forty refusal exemplars, iterated three times against the author’s corrections until every decline matched the axis of its question, moved honesty from 4/3/1 to 7/1/0. That is real transfer, and it is the strongest evidence in the lab that fine-tuning works when the data teaches a behavior. But assessment agreement stayed below the untuned base in both rounds, and the tuned models lost the advisory head-to-heads 13-0 and 12-1. Imitating judgment’s outputs did not produce judgment.

The control decided the lab. One paragraph of standing positions, the expert’s actual rules for performance claims, roadmap speculation, scale behavior, and framework structure, written into the system prompt of the untuned base, scored 8/0/0 on honesty, 3/0/0 on discipline, and the best placement agreement of any run on the box. It cost nothing to build and it beat three hours of training on curated data. The same block then transferred, unmodified, to the production advisory service running Gemini 2.5 Pro on another cloud, and a follow-up gate run against the real prompt surfaces sharpened the finding. A legacy prompt failed half the honesty probes by inventing a performance winner and a vendor roadmap; the live assembled prompt, which already carries the author’s voice document, held honesty at 7 of 8 stock; and a bare fallback path fabricated worst, 3 of 8, inventing a throughput figure outright. The block took all three surfaces to 8 of 8 and reproduced the lab’s numbers exactly on a prompt ten times the size it was tuned on. Two iterations were needed: a scoping sentence to stop answers reciting frameworks on questions that did not raise them, and a scope exclusion for the research surface, whose mission is finding pricing and performance data and whose axis the block contradicts. A constraint written for one mission does not paste onto another, and the gate is what caught that before deploy.

A bit-rate sub-bench answered the box’s serving question with the same gate. NVIDIA’s official 4-bit quant of the 26B matched bf16 on every quality family at a third the memory and 25 percent more single-stream speed, and the naive expectation of a 4x bandwidth win did not survive contact: active-parameter decode is not purely weights-bound on this architecture. Getting the measurement also surfaced a platform finding. Every 4-bit path on the box was blocked by month-old software, the cached serving container predating the model architecture, the local build predating the quant format, and the GGUF runtime predating both, until the current month’s container fixed all of it. On a platform whose pitch is 4-bit inference, the update treadmill is part of the product.

In 4+1 terms, the industry instinct this lab tested is a placement error: it tries to solve a Layer 2C problem at Layer 2B, compiling reasoning-plane policy down into the runtime artifact. The compilation failed, and the failure clarified the map. Judgment, the standing positions, and the validator are 2C functions and stay explicit. The harms they prevent are Layer 3 incidents, where the buyer meets the product. And with 2C retained as text, the 2B model drops to a substitutable component, proven by the same constraint block governing two model families on two clouds. The arc of the three labs is now one sentence per lab: managed retrieval hides the chunking decision, managed fine-tuning keeps the weights, and the weights were never where the judgment lived. The expert system that emerged is a stock model, a retrieval corpus, a paragraph of constraints, and a gate that regression-tests all of it, every piece plain text, every piece owned. That is a stronger ownership position than a fine-tuned artifact, because it survives the next model swap.

## Assessments at the Time of the Lab

| Vendor | Layer | Grade | As assessed |
| --- | --- | --- | --- |
| NVIDIA AI Platform | Layer 0 · Compute | NVIDIA Strength — Silicon Authority | May 22, 2026 |
| NVIDIA AI Platform | Layer 2B · Runtime | NVIDIA Authority — Inference + Agent Runtime | May 22, 2026 |
| Google Cloud AI Infrastructure | Layer 2B · Runtime | Ceded — Model-Integrated Stack | June 29, 2026 |

## Method and Disclosure

Self-funded, no sponsor. The candidates span an owned DGX Spark (Gemma 4 26B-A4B and Qwen3-4B, base and LoRA-tuned, bf16 and NVFP4) and the production advisory stack (Gemini 2.5 Pro on Vertex AI with the live system prompt), all scored by the same gate.

The gate: 46 assessment probes keyed to published vendor rulings held out of training, 8 honesty probes across three stance axes, 3 framework-discipline probes, a mechanical repetition screen, and 13 advisory head-to-heads judged against the author’s real published answers, with a human blind read as the final screen. Training was LoRA in the NGC PyTorch container; serving and judging ran on vLLM.

The gate design, scores, configs, and cost shape ship, including the raw-detail download. The probe contents and answer keys, the training pairs, the stance block’s full production text, and the corpus stay proprietary. Returns, not algorithms.

---
*Layer2C Labs · The CTO Advisor LLC · labs.layer2c.com*