Fine-tune the interpreter, then find out what it does to the queue.
By Keith Townsend · October 20, 2026
In a Deterministic Code In The Loop (DCITL) workflow, code runs the conversation and makes every decision, and a model only answers closed questions code can't parse: did the patient accept the 2pm slot, did they choose to pay out of pocket? Lab 23 measured what those answers cost: misread consent booked appointments nobody agreed to, and when code decided consent itself, patients went to a human instead. So can a small fine-tune (a LoRA adapter, a few megabytes trained on top of the model) read those messages better than a prompt carrying worked examples? What does it do to the staff queue? And when a managed platform does the tuning, is the adapter yours to run anywhere?
One clinic booking workflow with scripted, non-interactive synthetic patients in one synthetic world (8 patients, 6 providers, 42 slots). One base model, Gemma 4 26B-A4B, a mixture-of-experts (MoE) model. 1,000 frozen test conversations (955 scored after 45 disputed) and a 300-conversation control set written by a different model (294 scored). One seed per scenario. CoreWeave Forge on its standard trial credits, October 2026.
Do Keep consent decided in code first, then tune. Where the model's "yes" counted, the tune cut consent misreads from 13 to 0 or 1 of 496. Where code decides consent, no arm misread any; the tune's gain there was 12 fewer avoidable hand-offs, all on the self-pay question, at a fifth of the tokens.
Do Count the whole queue, not just the avoidable part, and measure it against more than one prompt. The tune sent 155 of 955 conversations to staff. The fair prompt sent 141; the same prompt with four mislabeled examples fixed sent 159.
Don’t Read hand-off counts as patient outcomes. Scripted patients can't confirm, call back or leave, so this lab can't say which routing serves them better.
Don’t Trust a hosted LoRA endpoint that returns HTTP 200 as proof your fine-tune is live. On this adapter, Forge's shared endpoint returned base-model output and no error.
Disclosure: Google Cloud is a client of this practice. Three Google models had seats here: Gemma, the subject; Gemini 3.8 Flash, which checked the scenarios; and Gemini 4 Argon, which wrote the control set and reviewed this entry. CoreWeave isn't a client and didn't see this lab before publish. The Futurum Group's Tech Field Day paid my travel and meals at AI Field Day 9, where I questioned CoreWeave on the portability claims tested here. The Forge work ran on the account's standard trial credits. OpenAI, NVIDIA, Apple and Anthropic, whose models, hardware and tools were used, aren't clients. The DGX Sparks and the Mac Studio were bought by the practice. No vendor commissioned, funded, previewed or had any say in this lab. No vendor paid for this answer.
A patient whose plan doesn't cover the visit says, "Don't worry about insurance, just book it." Has that patient chosen to pay out of pocket? No. Base Gemma 4 26B-A4B thought so. In this lab it answered that self-pay question correctly 13% of the time, and when its "yes" counted, it booked appointments nobody agreed to.
Lab 23 found that problem in a clinic booking agent built the Deterministic Code In The Loop (DCITL) way: code runs the conversation and makes every decision, and a model only answers the closed questions code can't parse. This lab asked the next question. Should the model reading those messages be fine-tuned, or just given better instructions? My earlier labs kept pointing at the prompt. Lab 6 found a paragraph beat 563 LoRA pairs. Then at AI Field Day 9, CoreWeave introduced me to Aria, the experiment agent in Forge, its managed fine-tuning platform, and it made me question that premise. I asked CoreWeave two things about Forge: do I get my LoRA, and can I run it on the shared base model at commodity rates? My bigger question was whether any of it holds on a mixture-of-experts model like Gemma 4 26B-A4B. This lab tested the portability half. I didn't run Aria.
So I trained small LoRA adapters, a few megabytes of extra weights on top of the model, two on an owned DGX Spark and one on Forge. The bar, set before the run, was a fair prompt carrying six worked examples per question. The tune had to beat it, paired on the same conversations and significant.
Did the tune beat the prompt?
Yes, on both halves of the bar. Read them separately, because they mean different things. Where the model's "yes" counted, every adapter cut consent misreads from 13 to 0 or 1 of 496 pressure conversations. Where code decides consent, the design that ships, no arm misread any. Code was already catching them. The tune's gain there was avoidable hand-offs to staff, from 16 to 4 of 955, and all 12 of the difference were the self-pay question. It did that at 285 tokens a conversation against the prompt's 1,468.
Look at how it got those 12. The tune answered "no", which ends the request. The scripted patient pushed again, met the same self-pay question, and the four-turn script ran out. The prompt said "unclear" twice and handed the patient to a person. And the training data never once had a patient say "yes" to self-pay in words the driver missed. I can't tell you how the tune treats a patient who means it.
Then why does the queue depend on the prompt?
Against the fair prompt, the tune sent more people to staff overall: 155 of 955 against 141. Most of the difference was one kind of request. 24 patients said, in effect, "I already confirmed with the front desk, just book the first available." The prompt picked the first open slot and asked them to confirm. The tune called it unclear, twice, and handed off. In this design a picked slot is only a proposal. Nothing books until the patient says yes in words code reads. A receptionist would probably book the first available and confirm.
Then the reviewers found problems in my fair prompt. One self-pay example labeled "Yes, definitely." as unclear, and two provider examples named a doctor nothing on screen identified. So I fixed them and reran it. It got worse: 34 avoidable hand-offs instead of 16, and 159 to staff instead of 141, level with the tune. Removing one example moved 18 conversations. The tune beat both prompts on the bar. The queue difference belonged to one prompt, not to tuning.
Which routing serves patients better? I don't know, and this data can't say. A scripted patient never answers the confirmation, never calls back, never gives up and goes elsewhere. Answering it takes patients who react, and that's a different lab.
Is the adapter yours?
Forge's training held up better than the pitch. 31 minutes, no memory tuning, $1.51 including a canary, against 2.2 hours per adapter on my Spark. The adapter downloads as a standard file.
Running it elsewhere took work. Stock vLLM won't load it, because Forge stores each layer's 128 experts as one fused tensor. That's a gap across the ecosystem's support for LoRA on mixture-of-experts models, not Forge's alone. Unsloth ships its own converter for it. I merged the adapter into the base model on a Spark, and translated it into a LoRA for mlx-lm on an M5 Ultra Mac Studio. Both gave the same outcome on all 1,910 frozen-corpus conversations. So yes, I got my LoRA, after engineering Forge doesn't provide or document.
The shared endpoint is the result I'd want a customer to see. Call the address of the adapter Forge trained and you get base Gemma: 8 or 9 of 10 answers identical to base, HTTP 200, no warning. I uploaded my own rank-16 adapter through Forge's bring-your-own path, and it ran, matching self-hosting on 1,909 of 1,910 conversations for five cents. The endpoint serves LoRAs on this model. It didn't serve the one Forge's own training produced, and it didn't say so. I only caught it because the grader checks answers, not responses.
So what?
Tune the interpreter, but keep consent decided in code first, and count the whole queue against more than one prompt. Removing one example from a prompt moved 18 conversations into the staff queue here. If you fine-tune on a managed platform, will your eval check the answers, or only that the endpoint said 200?
The numbers
Where each layer belongs
Read for portability. Could you take this layer elsewhere without rebuilding? How to read this table
| Layer | Placement |
|---|---|
Layer 2C · Reasoning Agentic Infrastructure — The Reasoning Plane interpreter, locally tuned adapter. Open weights, standard PEFT adapter, served on stock vLLM. | Retained |
Layer 2C · Reasoning Agentic Infrastructure — The Reasoning Plane interpreter, Forge-trained adapter. The file downloads; running it off Forge took a merge or translation Forge doesn't provide or document. | Retained |
Layer 2C · Reasoning Agentic Infrastructure — The Reasoning Plane Forge shared endpoint, our uploaded LoRA. Ran at commodity rates and matched self-hosting. | Delegated |
Layer 2C · Reasoning Agentic Infrastructure — The Reasoning Plane Forge shared endpoint, Forge's own adapter. Returned base output for it with no error; you can't verify or change what's served on that path (proposed after round-2 review). | Ceded |
Layer 2B · Runtime Application Runtime & Execution DCITL driver and consent gate. Plain Python the practice owns; placed at Layer 2B as in Lab 23. | Retained |
Layer 0A · Compute Compute DGX Spark, vLLM. NVIDIA hardware, open runtime. | Retained |
Layer 0A · Compute Compute Mac Studio, mlx-lm. The workload isn't tied to Apple hardware: the same adapter gave the same outcomes on a Spark. Follows Lab 21's Mac row (changed from Ceded after round-3 review). | Retained |
Sources
Method and disclosure
Editorial and self-funded. Harness: Lab 23's frozen DCITL driver, scorer and 1,000-scenario corpus, untouched. Training scenarios: the same generator with new seeds, deduplicated by content against the test set, written by GPT-5.6-Sol and checked blind by Gemini 3.8 Flash; the control set written by Gemini 4 Argon. Labels from a replay of the frozen driver with an answer key. Local training: Lab 18's LoRA recipe on a DGX Spark. Forge: Serverless SFT on standard trial credits, adapter pulled through OpenPipe ART 0.5.20. Serving: vLLM 0.25.1 on Sparks, mlx-lm 0.32.2 on an owned M5 Ultra Mac Studio, Forge Serverless Inference. Adversarial reviews: Gemini 4 Argon through Antigravity, and GPT-5.6-Sol through Codex. Spend: $17.58 metered ($16.02 OpenAI, $1.51 Forge training, $0.05 Forge inference). The harness, generator fork, label oracle, example pickers, merge and translation scripts are the instrument and can be published. The verdict and the trained adapters stay proprietary. Community context: huggingface/peft#3874; vllm-project/vllm#43254, #50252, #52198, #53470, #61453; unslothai/unsloth-zoo#603.
The essay on this page was drafted by Claude Opus 5.5, an Anthropic model, from the lab’s frozen structured record under this practice’s voice specification, checked by a deterministic gate against every figure in the record, and validated by the author before publication. The record is the canonical surface: where prose and record disagree, the record rules.
Download the raw lab detail (Markdown)