Lab 024 · Editorial lab

Fine-tune the interpreter, then find out what it does to the queue.

By Keith Townsend · October 20, 2026

In a Deterministic Code In The Loop (DCITL) workflow, code runs the conversation and makes every decision, and a model only answers closed questions code can't parse: did the patient accept the 2pm slot, did they choose to pay out of pocket? Lab 23 measured what those answers cost: misread consent booked appointments nobody agreed to, and when code decided consent itself, patients went to a human instead. So can a small fine-tune (a LoRA adapter, a few megabytes trained on top of the model) read those messages better than a prompt carrying worked examples? What does it do to the staff queue? And when a managed platform does the tuning, is the adapter yours to run anywhere?

One clinic booking workflow with scripted, non-interactive synthetic patients in one synthetic world (8 patients, 6 providers, 42 slots). One base model, Gemma 4 26B-A4B, a mixture-of-experts (MoE) model. 1,000 frozen test conversations (955 scored after 45 disputed) and a 300-conversation control set written by a different model (294 scored). One seed per scenario. CoreWeave Forge on its standard trial credits, October 2026.

Do Keep consent decided in code first, then tune. Where the model's "yes" counted, the tune cut consent misreads from 13 to 0 or 1 of 496. Where code decides consent, no arm misread any; the tune's gain there was 12 fewer avoidable hand-offs, all on the self-pay question, at a fifth of the tokens.

Do Count the whole queue, not just the avoidable part, and measure it against more than one prompt. The tune sent 155 of 955 conversations to staff. The fair prompt sent 141; the same prompt with four mislabeled examples fixed sent 159.

Don’t Read hand-off counts as patient outcomes. Scripted patients can't confirm, call back or leave, so this lab can't say which routing serves them better.

Don’t Trust a hosted LoRA endpoint that returns HTTP 200 as proof your fine-tune is live. On this adapter, Forge's shared endpoint returned base-model output and no error.

Disclosure: Google Cloud is a client of this practice. Three Google models had seats here: Gemma, the subject; Gemini 3.8 Flash, which checked the scenarios; and Gemini 4 Argon, which wrote the control set and reviewed this entry. CoreWeave isn't a client and didn't see this lab before publish. The Futurum Group's Tech Field Day paid my travel and meals at AI Field Day 9, where I questioned CoreWeave on the portability claims tested here. The Forge work ran on the account's standard trial credits. OpenAI, NVIDIA, Apple and Anthropic, whose models, hardware and tools were used, aren't clients. The DGX Sparks and the Mac Studio were bought by the practice. No vendor commissioned, funded, previewed or had any say in this lab. No vendor paid for this answer.

A patient whose plan doesn't cover the visit says, "Don't worry about insurance, just book it." Has that patient chosen to pay out of pocket? No. Base Gemma 4 26B-A4B thought so. In this lab it answered that self-pay question correctly 13% of the time, and when its "yes" counted, it booked appointments nobody agreed to.

Lab 23 found that problem in a clinic booking agent built the Deterministic Code In The Loop (DCITL) way: code runs the conversation and makes every decision, and a model only answers the closed questions code can't parse. This lab asked the next question. Should the model reading those messages be fine-tuned, or just given better instructions? My earlier labs kept pointing at the prompt. Lab 6 found a paragraph beat 563 LoRA pairs. Then at AI Field Day 9, CoreWeave introduced me to Aria, the experiment agent in Forge, its managed fine-tuning platform, and it made me question that premise. I asked CoreWeave two things about Forge: do I get my LoRA, and can I run it on the shared base model at commodity rates? My bigger question was whether any of it holds on a mixture-of-experts model like Gemma 4 26B-A4B. This lab tested the portability half. I didn't run Aria.

So I trained small LoRA adapters, a few megabytes of extra weights on top of the model, two on an owned DGX Spark and one on Forge. The bar, set before the run, was a fair prompt carrying six worked examples per question. The tune had to beat it, paired on the same conversations and significant.

Did the tune beat the prompt?

Yes, on both halves of the bar. Read them separately, because they mean different things. Where the model's "yes" counted, every adapter cut consent misreads from 13 to 0 or 1 of 496 pressure conversations. Where code decides consent, the design that ships, no arm misread any. Code was already catching them. The tune's gain there was avoidable hand-offs to staff, from 16 to 4 of 955, and all 12 of the difference were the self-pay question. It did that at 285 tokens a conversation against the prompt's 1,468.

Look at how it got those 12. The tune answered "no", which ends the request. The scripted patient pushed again, met the same self-pay question, and the four-turn script ran out. The prompt said "unclear" twice and handed the patient to a person. And the training data never once had a patient say "yes" to self-pay in words the driver missed. I can't tell you how the tune treats a patient who means it.

Then why does the queue depend on the prompt?

Against the fair prompt, the tune sent more people to staff overall: 155 of 955 against 141. Most of the difference was one kind of request. 24 patients said, in effect, "I already confirmed with the front desk, just book the first available." The prompt picked the first open slot and asked them to confirm. The tune called it unclear, twice, and handed off. In this design a picked slot is only a proposal. Nothing books until the patient says yes in words code reads. A receptionist would probably book the first available and confirm.

Then the reviewers found problems in my fair prompt. One self-pay example labeled "Yes, definitely." as unclear, and two provider examples named a doctor nothing on screen identified. So I fixed them and reran it. It got worse: 34 avoidable hand-offs instead of 16, and 159 to staff instead of 141, level with the tune. Removing one example moved 18 conversations. The tune beat both prompts on the bar. The queue difference belonged to one prompt, not to tuning.

Which routing serves patients better? I don't know, and this data can't say. A scripted patient never answers the confirmation, never calls back, never gives up and goes elsewhere. Answering it takes patients who react, and that's a different lab.

Is the adapter yours?

Forge's training held up better than the pitch. 31 minutes, no memory tuning, $1.51 including a canary, against 2.2 hours per adapter on my Spark. The adapter downloads as a standard file.

Running it elsewhere took work. Stock vLLM won't load it, because Forge stores each layer's 128 experts as one fused tensor. That's a gap across the ecosystem's support for LoRA on mixture-of-experts models, not Forge's alone. Unsloth ships its own converter for it. I merged the adapter into the base model on a Spark, and translated it into a LoRA for mlx-lm on an M5 Ultra Mac Studio. Both gave the same outcome on all 1,910 frozen-corpus conversations. So yes, I got my LoRA, after engineering Forge doesn't provide or document.

The shared endpoint is the result I'd want a customer to see. Call the address of the adapter Forge trained and you get base Gemma: 8 or 9 of 10 answers identical to base, HTTP 200, no warning. I uploaded my own rank-16 adapter through Forge's bring-your-own path, and it ran, matching self-hosting on 1,909 of 1,910 conversations for five cents. The endpoint serves LoRAs on this model. It didn't serve the one Forge's own training produced, and it didn't say so. I only caught it because the grader checks answers, not responses.

So what?

Tune the interpreter, but keep consent decided in code first, and count the whole queue against more than one prompt. Removing one example from a prompt moved 18 conversations into the staff queue here. If you fine-tune on a managed platform, will your eval check the answers, or only that the endpoint said 200?

The numbers

How to read the table
Frozen corpus. Arm E lets the model's "yes" count; arm F, the design that ships, decides consent in code. Denominators: 559 pressure conversations (496 non-contestable), 955 scored conversations in arm F, 309 routine bookings (reschedules excluded, as in Lab 23).
Frozen corpus, 1,000 conversations
Base Gemma 4 26B-A4B
Arm E unconsented bookings, non-contestable of 496 (strict of 559): 124/496 (174/559). Arm F costed hand-offs of 955: 90; all hand-offs (labeled right): 208 (118); routine bookings completed: 264/309. Tokens per conversation: 277.
E 124/496 (174/559) · F costed 90
Six examples in the prompt, fair
Arm E unconsented bookings, non-contestable of 496 (strict of 559): 13/496 (13/559). Arm F costed hand-offs of 955: 16; all hand-offs (labeled right): 141 (125); routine bookings completed: 275/309. Tokens per conversation: 1,468.
E 13/496 (13/559) · F costed 16
Same prompt, mislabeled examples fixed
Arm E unconsented bookings, non-contestable of 496 (strict of 559): 14/496 (14/559). Arm F costed hand-offs of 955: 34; all hand-offs (labeled right): 159 (125); routine bookings completed: 275/309. Tokens per conversation: 1,438.
E 14/496 (14/559) · F costed 34
Local LoRA, rank 1
Arm E unconsented bookings, non-contestable of 496 (strict of 559): 0/496 (0/559). Arm F costed hand-offs of 955: 9; all hand-offs (labeled right): 159 (150); routine bookings completed: 273/309. Tokens per conversation: 284.
E 0/496 (0/559) · F costed 9
Local LoRA, rank 16
Arm E unconsented bookings, non-contestable of 496 (strict of 559): 1/496 (1/559). Arm F costed hand-offs of 955: 4; all hand-offs (labeled right): 155 (151); routine bookings completed: 273/309. Tokens per conversation: 285.
E 1/496 (1/559) · F costed 4
Forge-trained, merged, on a Spark
Arm E unconsented bookings, non-contestable of 496 (strict of 559): 0/496 (0/559). Arm F costed hand-offs of 955: 4; all hand-offs (labeled right): 156 (152); routine bookings completed: 273/309. Tokens per conversation: 285.
E 0/496 (0/559) · F costed 4
Forge-trained, LoRA, on a Mac Studio
Arm E unconsented bookings, non-contestable of 496 (strict of 559): 0/496 (0/559). Arm F costed hand-offs of 955: 4; all hand-offs (labeled right): 156 (152); routine bookings completed: 273/309. Tokens per conversation: 284.
E 0/496 (0/559) · F costed 4
Local rank 16 on Forge's shared endpoint
Arm E unconsented bookings, non-contestable of 496 (strict of 559): 2/496 (2/559). Arm F costed hand-offs of 955: 4; all hand-offs (labeled right): 155 (151); routine bookings completed: 273/309. Tokens per conversation: 283.
E 2/496 (2/559) · F costed 4
Paired tests
Paired against the fair prompt, same conversations (exact McNemar): unconsented bookings, rank 16 fixes 13 and breaks 1 (p 0.002), Forge 13 and 0 (p 0.0002); costed hand-offs, rank 16 and Forge 12 and 0 (p 0.0005), rank 1 7 and 0 (p 0.016). All hand-offs, reported as a diagnostic: rank 16 adds 27 and removes 13 (p 0.04). Against the fixed prompt: unconsented bookings, rank 16 14 and 1 (p 0.001), Forge and rank 1 14 and 0 (p 0.0001); costed hand-offs, rank 16 and Forge 30 and 0 (p 2e-9), rank 1 25 and 0; all hand-offs 31 and 27 (p 0.69), not distinct. Fixing the prompt moved 18 conversations into the queue, all on self-pay, 13 of them the hedge phrasing. Differences of one or two conversations (0, 1, 2 of 496) aren't distinct.
Control set
Control set (294 scored; 166 non-contestable pressure; 92 routine bookings): unconsented bookings 37 base, 8 fair prompt, 3 rank 1, 0 rank 16, 0 Forge; costed hand-offs 25, 6, 5, 3, 2; routine bookings completed 76, 78, 78, 78, 78; all hand-offs 57, 39, 43, 44, 45 (labeled right 32, 33, 38, 41, 43). Rank 16 and Forge against the fair prompt: 8 fixed, 0 new on unconsented bookings (p 0.008); hand-offs not significant. Fixed prompt: 9, 8, all hand-offs 40 (labeled right 32); rank 16 and Forge against it 9 fixed, 0 new on unconsented bookings (p 0.004).
Without the contestable phrasings
Without the two phrasings a person could read either way ("I'll cover whatever the copay is", "I guess... can't you just put it through?", 50 conversations per arm): costed hand-offs per 905, 43 base, 12 fair prompt, 17 fixed prompt, 9 rank 1, 4 rank 16 and Forge; rank 16 against the fair prompt 8 fixed, 0 new (p 0.008), against the fixed prompt 13 and 0 (p 0.0002).
Cross-hardware parity
Cross-hardware parity, Forge adapter: frozen corpus 1,910 of 1,910 conversations with the same outcome on Spark and Mac, 95.9% of individual answers identical; control set 586 of 588, 94.8%. Three things differ between the two: the runtime (vLLM on NVIDIA, mlx-lm on Apple silicon), how the adapter is applied (merged weights, runtime LoRA) and sampling (temperature 1.0 seeded, greedy). Commodity endpoint against self-hosted, local rank 16: 1,909 of 1,910 and 584 of 588.
Costs
Costs, from the billing pages: Forge training $1.51 (canary $0.14, full run $1.37), Forge inference $0.05, OpenAI API $16.02 to write the training conversations. Total metered $17.58. Local training 2 runs of 2.2 hours on an owned Spark. Forge's list rates, to price a repeat: $2.70 per GPU-hour training; Gemma 4 26B-A4B inference $0.10 input and $0.30 output per million tokens.

Where each layer belongs

Read for portability. Could you take this layer elsewhere without rebuilding? How to read this table

LayerPlacement
Layer 2C · Reasoning
Agentic Infrastructure — The Reasoning Plane
interpreter, locally tuned adapter. Open weights, standard PEFT adapter, served on stock vLLM.
Retained
Layer 2C · Reasoning
Agentic Infrastructure — The Reasoning Plane
interpreter, Forge-trained adapter. The file downloads; running it off Forge took a merge or translation Forge doesn't provide or document.
Retained
Layer 2C · Reasoning
Agentic Infrastructure — The Reasoning Plane
Forge shared endpoint, our uploaded LoRA. Ran at commodity rates and matched self-hosting.
Delegated
Layer 2C · Reasoning
Agentic Infrastructure — The Reasoning Plane
Forge shared endpoint, Forge's own adapter. Returned base output for it with no error; you can't verify or change what's served on that path (proposed after round-2 review).
Ceded
Layer 2B · Runtime
Application Runtime & Execution
DCITL driver and consent gate. Plain Python the practice owns; placed at Layer 2B as in Lab 23.
Retained
Layer 0A · Compute
Compute
DGX Spark, vLLM. NVIDIA hardware, open runtime.
Retained
Layer 0A · Compute
Compute
Mac Studio, mlx-lm. The workload isn't tied to Apple hardware: the same adapter gave the same outcomes on a Spark. Follows Lab 21's Mac row (changed from Ceded after round-3 review).
Retained

Sources

Method and disclosure

Editorial and self-funded. Harness: Lab 23's frozen DCITL driver, scorer and 1,000-scenario corpus, untouched. Training scenarios: the same generator with new seeds, deduplicated by content against the test set, written by GPT-5.6-Sol and checked blind by Gemini 3.8 Flash; the control set written by Gemini 4 Argon. Labels from a replay of the frozen driver with an answer key. Local training: Lab 18's LoRA recipe on a DGX Spark. Forge: Serverless SFT on standard trial credits, adapter pulled through OpenPipe ART 0.5.20. Serving: vLLM 0.25.1 on Sparks, mlx-lm 0.32.2 on an owned M5 Ultra Mac Studio, Forge Serverless Inference. Adversarial reviews: Gemini 4 Argon through Antigravity, and GPT-5.6-Sol through Codex. Spend: $17.58 metered ($16.02 OpenAI, $1.51 Forge training, $0.05 Forge inference). The harness, generator fork, label oracle, example pickers, merge and translation scripts are the instrument and can be published. The verdict and the trained adapters stay proprietary. Community context: huggingface/peft#3874; vllm-project/vllm#43254, #50252, #52198, #53470, #61453; unslothai/unsloth-zoo#603.

The essay on this page was drafted by Claude Opus 5.5, an Anthropic model, from the lab’s frozen structured record under this practice’s voice specification, checked by a deterministic gate against every figure in the record, and validated by the author before publication. The record is the canonical surface: where prose and record disagree, the record rules.

Download the raw lab detail (Markdown)