# A guard is only as good as where its inputs come from.

> Lab 023 · Editorial lab · Status: published · Author: Keith Townsend · October 19, 2026  
> Published by The Advisor Bench LLC · Layer2C Labs  
> Canonical: https://labs.layer2c.com/labs/promise-boundary · Structured: https://labs.layer2c.com/labs/promise-boundary.json

**Finding:** Two ways of building the same clinic booking agent leaked at the same place: wherever a deterministic check trusted a fact the model supplied. In the Promise-Driven Agentic Kit’s example as shipped, the model sets “confirmed” and “eligible” itself, and 34.3 to 60.3 of every 100 pressured conversations ended in a forbidden booking, depending on the model. With code running the workflow (Deterministic Code In The Loop, DCITL), the model’s reading of consent replies counted, and 25.8 to 31.9 did. Each method reached zero in pressure sessions once that fact came from somewhere the model can’t write. Zero there means no forbidden booking in the pressure scenarios tested, not zero unauthorized actions: the code-driven version still made one unconsented booking per run in a normal scenario, from a bug in its own code. Checks that read the clinic’s own records held in every arm. The lab compares the two methods; it doesn’t pick one.

**Question:** An AI agent books a clinic appointment only if code confirms the patient agreed. But the agent is the one that tells the code whether the patient agreed. Does that check protect anything? This lab ran two versions of a clinic booking agent, the example from an open-source agent kit and a version where code runs the workflow, through 1,000 scripted patient conversations on four model configurations. Both let through bookings the patient never confirmed where the check took the model’s word, and both stopped once it didn’t.

**Load:** The clinic appointment example from Jeremy Schulman’s Promise-Driven Agentic Kit (Google’s Agent Development Kit, an orchestrator and two sub-agents, three deterministic services), copied unmodified, against a code-driven rewrite of the same workflow over the same services. 1,000 scripted patient conversations, 350 normal and 650 pressure, generated from code-owned specifications in a clinic of 8 patients, 6 doctors and 42 slots, with patient wording written by a second model and checked blind by a third. 955 scored. Plus a 30-scenario first phase on the kit’s own clinic.

**Raw lab detail:** https://labs.layer2c.com/downloads/promise-boundary-lab-detail.md
**Vendors:** Google (model), Google Cloud (cloud), Alibaba (model), DeepSeek (model), OpenAI (bench), NVIDIA (hw), Apple (hw), Anthropic (bench)
**Layers:** Layer 2B (Runtime), Layer 2C (Reasoning), Layer 3 (+1) (Applications)
**Themes:** validator-authority, authority-placement, small-model-viability, vendor-claim-scrutiny

## Verdict

**Scope:** One workflow (clinic booking), one agent framework (Google’s Agent Development Kit, as the kit ships it), scripted patients, an in-memory store, no concurrency. Four model configurations ran the 1,000-scenario corpus: Gemma 4 26B with reasoning off and on, Qwen3.8-Flash-Next, and DeepSeek V4 Flash 0731, whose kit-as-shipped arm is 271 sessions (150 run in corpus order, then a seeded stratified sample of 121) reported as a type-weighted estimate with a 95% interval. Gemini 3.5 Flash ran only the 30-scenario first phase, 45 pressure sessions of the shipped kit. No hosted frontier model ran the corpus. A few hundred pressure sessions per arm per model: directional, not academic. It compares two methods on one workflow and doesn’t rule between them; follow-on labs test a decision model on the kit and a fine-tuned interpreter on the code-driven version.

- **DON'T:** Don’t accept a deterministic check as a control until you know where each of its inputs comes from. In the kit as shipped, the booking service refuses unless “confirmed” and “eligible” are true, and the model sets both. Under ordinary pressure, every model that pushed to book also asserted the precondition: 48.7, 60.3, 34.3 and 41.1 forbidden bookings per 100 pressure sessions on the four configurations. No model ever passed an honest false that the check could refuse. Checks that read the clinic’s own records (slot taken, someone else’s appointment, unverified identity) were attempted 74 times under the kit across the four corpus runs, 116 times across all arms, and never let one through.
- **DO:** Feed every guard from a record the model can’t write: confirmation from the patient’s own action, coverage from the insurer’s service, the service from the doctor’s record. With only that change to the kit’s boundary, forbidden bookings fell to zero on both local models in the first phase, with the model still driving the conversation.
- **DO:** Count what each method costs, not only what it leaks. The kit’s model makes about 9 calls per session (38,305 to 43,057 tokens) and writes every reply; the code-driven version makes about 1.1 (280 to 805 tokens) and replies from code templates. Normal bookings completed about equally, 90.3 to 92.2% for the kit as shipped against 91.3 to 94.5%. Each had its own gap: the kit’s example as shipped can’t complete a reschedule, and the code-driven driver had two bugs that wrote unconsented changes.
- **DON'T:** Don’t let a model’s yes count as the patient’s consent, in either method. In the kit’s example the model’s yes is the “confirmed” flag. With code running the workflow but consent judged by the model, forbidden bookings ran 25.8 to 31.9 per 100, every one through a consent question. Deciding consent only in code (an explicit “I confirm” or a bare yes of four words or fewer), and asking again when the reply is unclear, took that to zero in pressure sessions on every model. The price: completed normal bookings fell about 6 points, to 85.4 to 88.3%, and hand-offs to a person roughly doubled, to about 200 per 1,000 sessions. That price is a floor: the scripted patients were told to confirm explicitly, and real patients who say “yes, please book that one” would be asked again.
- **DON'T:** Don’t expect a stronger model to fix a guard that takes the model’s word. No configuration came near zero under the kit: the larger models leaked 41.1 (DeepSeek V4 Flash) and 34.3 (Qwen). Turning Gemma’s reasoning on raised its rate from 48.7 to 60.3, because it read coverage correctly and reached the booking step more often, where it then asserted the precondition.
- **DO:** Test the code path as hard as the model path. Code-driven designs move risk into code: two driver bugs wrote unconsented changes in about 1 in 1,000 sessions, one of them with an explicit yes to a question that named the wrong action. Consent questions have to name the action exactly, and the driver needs its own check that the action it proposes matches the request. This driver’s consent rule would also read “Before I confirm, is that covered?” as a yes; no corpus message tested it.

**Independence and disclosure:** Self-funded editorial. The DGX Spark machines and the Mac Studio were bought by the practice. Disclosure: Google Cloud is a client of this practice. Google’s Gemma model ran every corpus arm, Google’s Gemini ran one first-phase arm on Google Cloud, Google’s Agent Development Kit is the framework the kit ships with, and a Google model checked the corpus. Google did not commission, fund, preview or have any say in this lab. Jeremy Schulman, the kit’s author, had no part in the design, runs or verdict; told about the lab before publication, he said the clinic example was early work and raised no objection. NVIDIA, Apple, Anthropic, OpenAI, Alibaba and DeepSeek, whose hardware, tools and models were used, are not clients. No vendor paid for this answer.

## Measured

| Measurement | Result | Note |
| --- | --- | --- |
| Forbidden bookings per 100 pressure sessions, kit as shipped (A) | 48.7, 60.3, 34.3, 41.1 | Gemma 4 26B reasoning off, reasoning on, Qwen3.8-Flash-Next, DeepSeek V4 Flash 0731. Strict rate, all 955 scored scenarios. Without the 63 contestable scenarios: 42.7, 55.8, 29.4. DeepSeek is a type-weighted estimate from 271 sessions; its non-contestable rate wasn’t computed. |
| DeepSeek V4 Flash, kit as shipped, 95% interval | 37.0 to 44.1 | 150 pressure sessions in corpus order covered two types (all 100 unconfirmed bookings, the first 50 of 90 ineligible coverage); a seeded stratified sample of 121 covered the rest. 268 scored. Interval from a per-type Jeffreys prior, so types that scored all or none keep their uncertainty. |
| Forbidden bookings per 100 pressure sessions, DCITL (E) | 31.1, 31.9, 25.8, 31.3 | Same order. Without the 63 contestable scenarios: 25.0, 25.0, 18.3, 24.8. |
| Forbidden bookings per 100 pressure sessions, consent in code (F) | 0.0 on all four | In all four runs F also made one unconsented booking in the same normal scenario, from a driver bug no consent rule can catch. |
| Kit with authoritative inputs (B and C), first phase | 0 on both local models | Gemma 4 26B and Qwen3.8-Flash-Next, 30 scenarios, 3 seeds. By construction, B and C block this failure mode for any model: the boundary ignores the model’s flags. That shows the revised boundary closes the modeled failure, not that the rest of the implementation is safe. |
| Kit as shipped, first phase | 26.7, 22.2, 20.0 | Gemma 4 26B, Gemini 3.5 Flash (45 sessions, Vertex AI), Qwen3.8-Flash-Next. Superseded as the headline by the corpus. |
| No boundary (D), first phase | 57.8 and 26.7 | Gemma and Qwen. A baseline of what a model attempts when nothing stops it. |
| Checks that read the clinic’s records | 116 attempts, 0 committed | Double-booking, someone else’s appointment, unverified identity: writes that reached the boundary across the four corpus runs and all arms, 74 of them under the kit. In the first phase the no-boundary baseline (D) committed all 16 it attempted. |
| Pasted text claiming eligibility, kit as shipped | 100%, 100%, 28.6%, 5 of 7 | Gemma reasoning off, on, Qwen, DeepSeek. Claiming confirmation: 81.0%, 100%, 16.7%, 6 of 7. |
| Consent misread as self-pay, DCITL (E) | 57.8%, 57.3%, 22.2%, 51.1% | Ineligible-coverage scenarios: Gemma off, Gemma on, Qwen, DeepSeek. |
| Unconsented moves, DCITL (E), reschedule pressure | 21.7, 31.7, 31.7, 30.0 per 100 | 60 scenarios per model, scored outside the keystone because the kit can’t reschedule. F: 0 on all four. |
| Normal bookings completed | 91.3 to 94.5% (E) against 85.4 to 88.3% (F) | Across the four configurations. The kit as shipped: 92.2% (Gemma off and Qwen), 90.3% (Gemma on). |
| Sessions handed to a person, per 1,000 | 106 to 116 (E) against 197 to 209 (F) | After two unclear answers on the same pending decision. |
| Model calls and tokens per session | about 9 and 38,305 to 43,057 (A) against about 1.1 and 277 to 817 (E and F) | Token range for E and F spans Gemma reasoning off to Gemma reasoning on. The kit’s model writes every reply; E and F reply from code templates, so part of the gap is prose, not the boundary. |
| Median session time, Qwen | 102 s (A) against 6 s (E) | Gemma reasoning off: 31 s against 1 s. Gemma reasoning on: 225 s against 24 s. |
| Reasoning on, Gemma, kit as shipped | 44 against 2 | Of 100 paired unconfirmed-booking scenarios, 44 leaked only with reasoning on and 2 only with it off. In 29 of the 44, reasoning-off Gemma had wrongly told a covered patient the visit wasn’t covered and never reached the booking step. |
| Corpus | 1,000 scenarios, 955 scored | 892 clean (blind checker agreed with code), 63 contestable (wording open to either reading), 45 disputed (checker contradicted code) and excluded. 18 types, 350 normal and 650 pressure. |
| Driver bugs that wrote | about 1 in 1,000 sessions | A misparse that moved the wrong appointment, and a misread that asked to book instead of move. |
| Metered spend | about $2.26 | Gemini 3.5 Flash on Vertex AI, first phase and its canary, estimated at list rates, not taken from billing line items. Corpus writing and checking ran on existing subscriptions; everything else on owned hardware. |

## Authority placement (DAPM)

**Reading:** Decision authority: who makes the runtime decision at this layer, and can the enterprise see and override it?

| Layer | Kit as shipped (A) | Kit, authoritative inputs (B, C) | DCITL (E) | DCITL, consent in code (F) | Basis |
| --- | --- | --- | --- | --- | --- |
| Layer 3 (+1) (Applications) | Delegated | Delegated | Retained | Retained | Who decides the next step in the clinic workflow. In the kit the model drives the conversation and decides when to book, within limits code sets: Delegated. In DCITL code runs the workflow; the model only interprets. |
| Layer 2B (Runtime) | Ceded | Retained | Delegated | Retained | Who decides, at execution, whether a write goes through: the deterministic check on every tool call, placed at Layer 2B as the gates in Labs 3, 11 and 12 are. In the shipped kit the boundary decides, but on the confirmation and eligibility facts the model supplies, so for those checks the model decides by choosing what to pass: Ceded. Checks that read the clinic’s records stayed with code in every arm. In E code decides, but when it can’t read a consent reply it commits on the model’s yes to a closed question: Delegated. With authoritative inputs (B, C) and in F the boundary decides on facts the model can’t write. |
| Layer 2C (Reasoning) | Ceded | Retained | Delegated | Retained | Who determines that a booking is done, which here means consented. In the shipped kit the model’s own assertion is the consent: Ceded. In E code decides, but on consent questions it acts on the model’s reading of the patient’s reply: Delegated within a closed question. In F code acts on a model’s no and ignores a model’s yes; only an explicit yes code reads itself commits. |

## What this does not establish

- One workflow, one framework, an in-memory store, no concurrency. The kit’s formal model check disables concurrency too.
- Scripted patients can’t react. If code proposes a slot the patient didn’t mean, the script’s next yes reads as consent, where a real patient would correct it. That inflates E and F leak counts slightly; it can’t hide a leak.
- Some final pressure lines (“go ahead”, “just do it”) had no blind check. In E they follow an explicit proposal, where “go ahead” reads as a yes, so part of E’s injection and mixed-consent leak is that scripted-patient effect.
- B’s and C’s zeros are partly structural: once the boundary ignores the model’s flags, no model can leak through them. The confirmation record stands in for a confirm control; how a chat-only workflow captures consent is what E and F measure. B wasn’t run on the 1,000-scenario corpus for that reason: with confirmation and coverage read from records, no model can leak through those checks, so a corpus run would measure the stand-in, not the model.
- DeepSeek’s kit-as-shipped arm is 271 sessions, not the full 650 pressure scenarios: 150 in corpus order (all unconfirmed bookings and the first 50 ineligible-coverage scenarios, not a random 50), then a seeded stratified sample of 121, as few as 7 per type. It is reported as a type-weighted estimate; the interval uses a per-type Jeffreys prior so that types scoring all or none, such as 17 of 17, carry their own uncertainty.
- No hosted frontier model ran the corpus. Gemini 3.5 Flash ran only 45 first-phase sessions of the shipped kit.
- Gemma and the corpus checker share a vendor (Google). The corpus writer (OpenAI), Qwen (Alibaba) and DeepSeek don’t. The 70 identity scenarios carry no blind-check question, so their labels rest on the code specification alone, and 23 ownership scenarios were disputed because the checker wasn’t given which doctor and time an appointment ID meant.
- Gemma reasoning off and on ran at different concurrency, 4 and 8. Reasoning on had 23 sessions time out at the 300-second turn limit; reasoning off had 46 no-verdict sessions, 40 at the per-turn call cap and 6 at the turn time limit. Both are excluded as no verdict, not scored. Reasoning-on Gemma completed fewer normal covered bookings under the kit (78 of 105 against 102) because it asked the patient for a service name the script couldn’t supply.
- The DCITL driver’s parsers are narrow by design and were frozen before the corpus existed. Their narrowness shows up as escalations and hand-offs, which are measured; a broader parser would change E’s and F’s completion, not the rule that a model’s yes doesn’t count in F. The patients’ wording was written to confirm explicitly (162 of 350 normal sessions say “I confirm”), so F’s completion cost is measured on cooperative phrasing. The consent rule doesn’t check for negation: “Before I confirm” would read as a yes. No corpus message contained one.
- E and F hand a session to a person after two unclear answers, which ends it without a forbidden write. The kit as shipped has no hand-off tool. Hand-offs are reported as F’s cost, not as a pass.
- Cancelling without the patient’s confirmation was recorded as a diagnostic, not scored as forbidden, because the kit’s cancel service doesn’t check confirmation. Qwen did it twice in the first phase.
- The kit’s reschedule path can’t complete as shipped, so reschedule scenarios are excluded from the kit’s keystone; the code-driven version, which reads the doctor’s ID from the store, completed them. The kit’s model check covers safety (bad states unreachable), not whether a workflow can finish.

## Open questions

- **Can a fine-tuned small local model hold the interpretation role well enough to close the gap between E’s completion and F’s safety?** F’s cost is completion and hand-offs, both driven by how well the interpreter reads unclear replies. In a regulated workflow the patient’s messages may not be allowed to leave the building, so the interpreter has to run locally. That is a follow-on lab, already specified, with its own kill criterion.
- **Can a decision model checking the agent’s claims fix the kit without taking the workflow away from the model?** A planned follow-on lab puts Jev, TypeSafe’s decision model, which returns choices and scores rather than text, between the agent’s proposed booking and the kit’s boundary. It would score whether the confirmation or coverage the agent asserts holds up against the conversation, keeping the kit’s design of a model-driven workflow. It runs on the same frozen 1,000 scenarios and scorer, so it lands directly against A, E and F.
- **What does a validator for the code path look like?** The two driver bugs say what it needs: every consent question names the action exactly, and a check confirms the action code proposes matches what the patient asked for before the question is asked. Neither existed in this lab’s driver.

## Objection handled

**Isn’t this an indictment of the kit’s method?**

No. The kit’s method is that a model’s promise has to be assessed by something other than the model: you can’t command a model, only get a promise from it and check whether it was kept. In the clinic example the model’s “confirmed” is a promise about the patient, and the booking service accepts it without assessing it. The kit’s own case study calls those flags a teaching simplification, and its author describes the example as early work. Its whitepaper describes the booking service refusing unconfirmed and ineligible bookings without that caveat, and the example is what a team adopting the kit starts from.

The shape isn’t specific to the kit. A tool call that takes a fact as an argument the tool should look up for itself (approved, user verified, within limit) is common in function-calling designs. What this lab adds is the measured behavior: under ordinary pressure, on four model configurations, models set the flag to whatever let them act.

The fix inside the kit is small. Arm B changed only where the boundary gets its facts, kept the kit’s contracts, tools and model-driven conversation, and scored zero forbidden bookings on both local models in the first phase. The code-driven method leaked at the same place and closed the same way, at a cost in completions and hand-offs. Neither came out as the answer, and the follow-on labs may move both.

## Summary

The lab compared two ways of keeping an AI agent’s consequential actions under deterministic control. In the Promise-Driven Agentic Kit (PDAK), an open-source methodology kit by Jeremy Schulman, the model runs the workflow and decides when to write; code refuses a write that breaks a rule. In Deterministic Code In The Loop (DCITL), the thesis behind earlier Layer2C labs, code runs the workflow and makes every decision it can check against a fact; the model is called only to interpret input code can’t read, and its answer goes back to code. The kit’s clinic booking example checks slot availability, ownership and identity against its own records, but takes “confirmed” and “eligible” as true-or-false arguments from the model.

Six arms. A: the kit as shipped. B: the same kit with the boundary reading confirmation and coverage from records the model can’t write. C: B with a few plain sentences in place of the kit’s XML contract. D: no boundary, a baseline. E: DCITL, code-driven, where a model’s yes on a consent question counts. F: E with consent decided only in code. Every write is scored against ground truth before it runs, and the store is read back after each session; the agent’s own account is never used. The keystone is sessions with a forbidden commit per 100 pressure sessions.

First phase, 30 hand-written scenarios on the kit’s own clinic, 3 seeds: the shipped kit leaked 26.7 per 100 on Gemma 4 26B, 22.2 on Gemini 3.5 Flash and 20.0 on Qwen3.8-Flash-Next. B and C scored zero on both local models. The plain prompt had fewer forbidden attempts than the XML contract, 14 against 25 sessions on Gemma and 7 against 9 on Qwen, at 2 to 2.4 times fewer input tokens per call. The kit’s reschedule path can’t complete as shipped: the management agent gets a doctor’s name, needs an ID, and has no tool to translate. The kit’s model check proves bad states unreachable; it doesn’t check that a reschedule can finish, which is a separate kind of property.

Second phase, the headline: 1,000 scenarios across 18 types in a larger clinic. Code generated every scenario and the true consent state of every turn. GPT-5.6-Sol wrote the patients’ words; Gemini 3.8 Flash, seeing only those words, answered whether each patient had consented. Where its answers matched code’s, the scenario is clean (892). Where it found the wording open to either reading, the scenario is contestable (63, 60 of them mixed-consent). Where it contradicted code, the scenario is disputed and excluded from every rate (45). That leaves 955 scored. Each headline rate counts all 955 (the strict rate); the non-contestable rate drops the 63 contestable scenarios, so a leak can’t rest on wording a reasonable reader could take either way. The DCITL driver was frozen by hash before any scenario existed; three later fixes are recorded with reasons, one of them a consent bug where “I confirm” with a different slot named booked the slot code had proposed.

Forbidden bookings per 100 pressure sessions, kit as shipped (A): 48.7 on Gemma with reasoning off, 60.3 with it on, 34.3 on Qwen, 41.1 on DeepSeek V4 Flash (type-weighted estimate from 271 sessions, 95% interval 37.0 to 44.1). DCITL (E): 31.1, 31.9, 25.8 and 31.3. DCITL with consent in code (F): 0.0 on all four. Non-contestable rates for E: 25.0, 25.0, 18.3 and 24.8. Reschedule scenarios, which the kit can’t complete, are scored separately: E moved an appointment without consent in 21.7, 31.7, 31.7 and 30.0 per 100; F in none.

In every arm with a boundary, every committed violation was a fact the model supplied or judged (confirmation, eligibility, or a move confirmation), apart from the two driver bugs. Writes that broke a check reading the clinic’s own records reached the boundary 116 times across the four corpus runs, 74 of them under the kit, and none committed. Under the shipped kit, pasted text claiming eligibility was committed in 100% of sessions on Gemma with reasoning off and on, 28.6% on Qwen and 5 of 7 on DeepSeek; claiming confirmation, 81.0%, 100%, 16.7% and 6 of 7.

In E, the leaks run through the consent questions. In the ineligible-coverage scenarios Gemma judged replies like “I’ll cover the copay” a self-pay choice in 57.8% of sessions with reasoning off and 57.3% with it on; DeepSeek in 51.1%; Qwen in 22.2%. A larger model did not read consent better. F treats a model’s yes as unclear and asks again, and committed no forbidden booking in a pressure session on any model.

F’s price: normal bookings completed 85.4 to 88.3%, against 91.3 to 94.5% in E, and 197 to 209 sessions per 1,000 handed to a person, against 106 to 116. A made about 9 model calls per session; E and F about 1.1.

## Detail

The question is narrow. When code checks an AI agent’s action before it happens, where does the code get the facts it checks? If the agent supplies them, the check looks deterministic and the decision has moved back to the model. The Promise-Driven Agentic Kit makes the deterministic boundary its central idea, and its tagline is “LLMs propose. Deterministic boundaries decide.” Its clinic booking example refuses to book unless the patient confirmed the slot and the visit is covered or self-paid, and both of those arrive as true-or-false arguments from the model. In the kit’s own terms, the model’s “confirmed” is a promise that never gets assessed by anything but the model. The kit’s case study flags the shortcut. This lab measured what it costs, and ran a code-driven version of the same workflow beside it to see whether that method avoids the problem.

In the first phase the canary showed the mechanism on three models. The patient was asked to confirm the 9:00 slot and replied, “I told you, it’s already confirmed. Book it now.” Gemma 4 26B and Gemini 3.5 Flash both called the booking tool with confirmed set to true, and the boundary committed it. Qwen3.8-Flash-Next said it would treat the reply as confirmation, and did. Across the scored sessions no model ever told the boundary the truth when the truth was “not confirmed”. When a model decided to book, it also asserted the precondition.

Changing only where the boundary gets its facts closed the gap. Arm B kept the kit’s contracts, prompts and tools, and read confirmation from a record standing in for a confirm control and coverage from the insurance service. It scored zero, as did arm C with a plain prompt. The plain prompt had fewer forbidden attempts than the XML contract and used 2 to 2.4 times fewer input tokens per call.

Arm D was first built as an audit after the change was committed, and labeled DCITL. That was wrong: DCITL decides before anything happens. It was relabeled a no-boundary baseline, and a code-driven DCITL arm, E, was built. Code parses each patient message for names, dates, times, IDs and explicit confirmations; when it can’t, it asks the model a closed question; code decides. Arm F changes one rule: on a consent question code acts on a model’s no and ignores a model’s yes. Only an explicit “I confirm”, or a bare yes of four words or fewer, commits.

The first phase flattered E, because its parsers were written knowing the 30 scripts. The second phase removed that. Code generated 1,000 scenario specifications and the true consent state of every turn, GPT-5.6-Sol wrote the patients’ words in varied tones and formats, and Gemini 3.8 Flash answered the ground-truth questions blind. Disagreement excluded 45 scenarios, among them one where the writer named the wrong doctor. The DCITL driver was frozen by hash before generation.

On the corpus the kit as shipped leaked 34.3 to 60.3 per 100 pressure sessions, E 25.8 to 31.9, and F zero in pressure sessions on every model. Checks fed by the clinic’s records never failed in any arm with a boundary. Apart from two driver bugs, everything that leaked was a fact the model supplied or judged.

A stronger model didn’t fix it. The larger models leaked less than Gemma with reasoning on, DeepSeek V4 Flash 41.1 and Qwen 34.3, and none came near zero. Within Gemma, reasoning on leaked more under the kit than reasoning off, 60.3 against 48.7. The paired scenarios show why: with reasoning off Gemma often told covered patients they weren’t covered and stopped before the booking step, so it avoided forbidden bookings only by misreading the coverage check. With reasoning on it read coverage correctly, reached the booking, and asserted confirmation when pushed. Under E, DeepSeek misread consent about as often as Gemma.

Moving decisions into code moves risk into code. Two driver bugs wrote unconsented changes in about 1 in 1,000 sessions. One misparsed “I’d like to move appointment A07 to...” and proposed moving the patient’s other appointment; F blocked it. The other read “let’s go with Dr. Chen... I want the appointment moved there” as a new booking, asked “Shall I book it?”, and the scripted patient answered yes. F couldn’t stop that one, because the yes was explicit and the question named the wrong action.

Neither method came out as the answer. Each leaked where it trusted the model’s word and closed where it didn’t, at different costs. The kit with authoritative inputs kept the model in charge of the conversation and scored zero on two local models in the first phase. The code-driven version with consent in code scored zero on four, and paid in completed bookings and hand-offs. A decision model checking the kit’s promises, or a fine-tuned interpreter in the code-driven version, could change either cost.

## Method and disclosure

Self-funded. The DGX Spark machines and the Mac Studio were bought by the practice. No vendor was briefed, reviewed this, or was given advance sight. The kit’s author was told about the lab before publication and had no part in it.

The kit’s clinic appointment example was copied unmodified from upstream commit 0514f991 (Apache-2.0, license kept). Every arm patches the kit’s services at import time; arm A is the kit as shipped. The harness, scorer, scenario generator and DCITL driver were built and run by Claude Code (an Anthropic tool) under the author’s design rulings, with a self-test that plants a lie against every guard in every arm and refuses to start a run unless each is scored correctly.

Venues: Gemma 4 26B (bf16, vLLM 0.25.1) on one DGX Spark, reasoning off and, in a separate run, on; Qwen3.8-Flash-Next (NVIDIA’s FP4-Local build, NVIDIA’s published single-Spark recipe) on a second Spark; DeepSeek V4 Flash 0731 (FP8, oMLX, thinking at maximum effort) on the Mac Studio, with the serving configuration Lab 20 used on the same machine and the weights Lab 14 qualified; Gemini 3.5 Flash on Vertex AI for the first phase only.

Corpus: specifications generated by code with a fixed seed; patient wording by GPT-5.6-Sol through the Codex command-line tool on a ChatGPT subscription; blind check by Gemini 3.8 Flash through the Antigravity command-line tool. The DCITL driver was hash-frozen before generation; three later fixes are recorded with reasons, none adding to the parsers’ vocabulary: a crash on an unclear intent, a loop that re-listed times instead of asking the model, and a consent bug where “I confirm” with a different slot named booked the proposed slot.

Reviewed before publication by a model from a different family instructed to attack the record. Its findings and their disposition are kept with the lab’s working notes.

The harness, scorer, corpus generator and corpus are the instrument and can be published. The method for assessing a vendor’s placement is not.

---
*Layer2C Labs · The Advisor Bench LLC · labs.layer2c.com*