{
  "slug": "who-decides-consent",
  "lab_number": 29,
  "title": "A four-word consent rule cost more clean bookings than swapping the model",
  "type": "editorial",
  "status": "published",
  "sponsor": null,
  "author": "Keith Townsend",
  "date": "October 24, 2026",
  "date_iso": "2026-10-24",
  "layers": [
    "layer2b",
    "layer2c",
    "layer3"
  ],
  "axes": [
    {
      "instrument": "4plus1",
      "layers": [
        "layer2b",
        "layer2c",
        "layer3"
      ],
      "has_dapm_table": true
    }
  ],
  "supersedes": [],
  "superseded_by": null,
  "corrects": [],
  "corrected_by": [],
  "vendors": [
    {
      "key": "google",
      "role": "model"
    },
    {
      "key": "anthropic",
      "role": "model"
    },
    {
      "key": "nvidia",
      "role": "hw"
    },
    {
      "key": "apple",
      "role": "hw"
    },
    {
      "key": "coreweave",
      "role": "cloud"
    },
    {
      "key": "openai",
      "role": "bench"
    }
  ],
  "themes": [
    "authority-placement",
    "validator-authority",
    "small-model-viability",
    "prompt-vs-weights",
    "vendor-claim-scrutiny"
  ],
  "finding": "On one simulated clinic, a code rule for consent cost an AI scheduling agent far more correct bookings than swapping the model that reads the patient. Lab 23’s rule counts only “I confirm” or a yes of four words or fewer. Letting the model’s yes count instead raised clean correct bookings (right, with no dependency left open) by 19 to 32 points with every reader; swapping Gemma 4 26B-A4B for Claude Haiku 5.5 moved them 3 points or less. It also brought hundreds of conversations per cell with a wrong write that the code rule didn’t have. Of the clean bookings the model-decides arm made and the code rule didn’t, 96 to 99% were clear yeses longer than four words, a polite “Yes, that’s right, please go ahead.” or a yes with urgency added, from patients who repeat their answer when asked again. Where the model decided, the readers failed differently: Gemma booked on an open time or doctor when the hedge took two or three questions, at every pressure level (7 to 27%); Haiku stayed under 6% in 11 of 12 settings and reached 10 to 13% in one, three questions with urgency alone. And under either rule, when a payment patient said they couldn’t decide yet, Haiku more often answered no, the driver ended the call, and Haiku got 22% of the patients who needed a person to one, against Gemma’s 34%.",
  "video": null,
  "lab_detail": "https://labs.layer2c.com/downloads/who-decides-consent-lab-detail.md",
  "sources": [
    {
      "label": "Promise-Driven Agentic Kit, by Jeremy Schulman (GitHub)",
      "url": "https://github.com/jeremyschulman/promise-driven-agentic-kit"
    },
    {
      "label": "clinicbench, the bench behind this lab (GitHub)",
      "url": "https://github.com/kltownsend/clinicbench"
    }
  ],
  "question": "A patient tells a clinic’s AI scheduling agent, “Yes, that’s right, please go ahead.” Is that a yes? To the code rule I tested, it’s seven words, and the rule only counts four or fewer. So who should decide whether a patient agreed: code, or the model? And does swapping in a frontier vendor’s model change the answer? I ran one agent against 1,000 simulated patients whose answers I controlled, swapped one part of its stack at a time, and scored every booking against what each patient actually meant.",
  "load": "Lab 23’s code-driven clinic scheduling driver (Deterministic Code In The Loop, DCITL: code runs the workflow and asks a model only what code can’t read), copied unchanged, against clinicbench, a new bench whose patients are code: each patient’s truth (what they’ll pay, when they can come, who they’ll see) is set before the call, and code decides what they say and how clearly. 1,000 patient truths over 8 patient identities and Lab 23’s 18 types, each at 48 settings of how unclear one thing is (payment, schedule or doctor): 48,000 conversations a cell. 22 cells on the entire set across two consent rules, three readers (Gemma 4 26B-A4B, Claude Haiku 5.5 with thinking off and on) and two prompts.",
  "verdict": {
    "scope": "One workflow (clinic scheduling), one code-driven driver (Lab 23’s, frozen), one code consent rule (Lab 23’s four-word rule), one simulated clinic of 8 patient identities, 6 doctors and 42 slots, and patients that are code. They react to which question the agent asks, but speak in templates and repeat their answer when asked again; they don’t shorten or rephrase. The readers are Gemma 4 26B-A4B with reasoning off on owned DGX Sparks and Claude Haiku 5.5 through Anthropic’s API, with thinking off and on. Each cell is 1,000 patient truths at 48 settings; the patients, not the conversations, are the independent observations. These are behaviors measured inside the simulation. They compare the parts here; they don’t show that the ordering holds with real callers, with a code rule that accepts a clear yes of any length, or with a patient who shortens a yes when asked again. Not yet run: that rule, that patient, Gemma with reasoning on, a model running the whole workflow (the next lab), and a model-written patient voice.",
    "independence": "Self-funded editorial. The DGX Sparks and the Mac Studio were bought by the practice. Disclosure: Google Cloud is a client of this practice, and Google’s Gemma is the reference reader. The driver’s prompts were written for Gemma in Lab 23, and the pre-freeze shakedown ran on Gemma; neither was tuned for Haiku. Anthropic is not a client; Claude Haiku 5.5 ran on API credits that come with a Claude plan I pay for. The bench was built with Claude Code, an Anthropic tool, which also wrote the patient’s template lines, so an Anthropic model wrote the words an Anthropic model reads. The scoring is deterministic code, never a model. CoreWeave, NVIDIA, Apple and OpenAI are not clients. No vendor saw, directed or paid for this work.",
    "calls": [
      {
        "kind": "do",
        "text": "Test your consent rule against how your callers actually say yes before you shop for a reader. On this bench, letting the model’s yes count instead of a four-word code rule raised clean correct bookings by 19 to 32 points with every reader (paired 95% intervals 16.0 to 34.5): 41% to 60% on payment, 14% to 43% on schedule, 24% to 50% on doctor, for Gemma. Counting every right booking, lucky guesses included, the gain was 21 to 37 points. Swapping Gemma for Haiku moved clean correct bookings by 3 points or less in every cell, in the single runs reported. The rule dominated the booking differences; the reader’s were small. Letting the model decide also brought hundreds of conversations per cell with a wrong write that the code rule didn’t have."
      },
      {
        "kind": "dont",
        "text": "Don’t let the model’s yes count without testing the reader on slow hedges and on pressed ones; these two failed on different settings. Counting only bookings made while the hedged time or doctor itself was still open, with the model deciding: when one question settled the hedge, both readers did it in 0.5% or fewer of those conversations. Gemma rose with the number of questions at every pressure level, 7 to 13% at two and 21 to 27% at three. Haiku stayed under 6% in 11 of the 12 settings of questions and pressure, and reached 10 to 13% in one: three questions with urgency alone (with urgency and a prop, 3 to 6%). Across each whole cell, Gemma had 526 and 750 conversations with a wrong write, Haiku 213 and 236. Under the code rule, no booking was made on an open time or doctor."
      },
      {
        "kind": "dont",
        "text": "Don’t send a reader’s no straight to a hang-up. This driver ends the call on no and hands off after unclear, under either consent rule. Asked whether to pay out of pocket, some patients say they can’t decide on this call: “I can’t tell you that right now. I’d have to check with my husband.” or “I honestly don’t know. I’d need to check and get back to you.” On payment under the code rule, Gemma reached a person and Haiku didn’t in 1,914 paired conversations, the reverse in 52. In 1,727 of the 1,914, Haiku had answered a self-pay question no and the driver never handed off; Gemma answered the same line unclear in 1,686 of them. Reached a person: Gemma 34%, Haiku 22%, a gap of 10 to 14 points in every arm and prompt. Thinking on didn’t close it, and borrowed worked examples widened it. Most patients who needed a person missed one under either reader; on schedule and doctor the readers were within 1.1 points."
      },
      {
        "kind": "dont",
        "text": "Don’t borrow worked examples without measuring them on your own workload. Lab 024’s six examples per question, sent as prior chat turns, moved three things on payment. Clean correct bookings rose a little (Gemma 11,281 to 11,585 under the code rule, 16,599 to 16,911 with the model deciding; Haiku by 32 and 46). On self-pay hedges, where four of five examples say no, Haiku moved further toward no and reached fewer patients who needed a person (22% to 20%), while Gemma stayed put. And with the model deciding, conversations with a wrong write rose on both readers, Gemma 453 to 851, Haiku 400 to 596, of 48,000. A small gain in bookings for a large rise in wrong writes. That’s these examples on payment, not a rule about few-shot."
      },
      {
        "kind": "do",
        "text": "Price the code rule against how your callers say yes. Lab 23’s rule counts “I confirm” or a yes of four words or fewer. A patient who talks at length says “Yes, that’s right, please go ahead.” (seven words); a patient pressed on the time or the doctor adds urgency, as in “Yes, book it. I really need this soon.” (eight). Either way the driver asks again, the patient repeats it, and the driver hands off. Of the conversations the model-decides arm booked cleanly and the code arm didn’t, 96 to 99% went that way in every cell; the code arm booked cleanly where the other didn’t in 15 to 246 per cell. On payment it’s almost all the long form, because urgency attaches to the dimension under test and payment’s confirmation doesn’t carry it; that’s part of why payment’s gap is the smallest. This bench’s patient never shortens on the second ask, so this is the rule’s cost against these patients, not against callers who would just say “Yes.”"
      }
    ]
  },
  "objection": {
    "q": "Isn’t a simulated patient too clean to tell you anything about real ones?",
    "a": [
      "It is clean, on purpose, and that’s where the authority comes from. Each patient’s truth is set in code before the call, code decides how clearly it’s stated at each setting, and every booking is scored against that truth by code. No model grades anything, and the same conversation scores the same way every time. A real call can’t give you that: you’d never know for certain what the caller meant.",
      "What the simulation can’t give you is the distribution of real callers. It doesn’t say how often patients hedge, how they phrase a yes, or whether they shorten it when asked again. So the lab doesn’t claim a production rate. It claims an ordering inside a controlled field, with paired intervals: changing the consent rule moved clean correct bookings by 19 to 32 points, and changing the reader moved them by 3 or less. The size of that gap is a property of this patient mix and this four-word rule; a rule that accepts a clear yes of any length, or callers who shorten, would narrow it.",
      "The patient’s simplicity cuts both ways, and hardest against the code rule. The patient repeats a long yes when asked again; a real caller asked for a plain yes or no might just say “Yes.” So the code rule’s cost here is likely higher than it would be with those callers. The reader comparison has the opposite limit: a clear yes comes from four fixed templates every reader handles, so the readers differ mostly on hedges, which is where this lab found them differing."
    ]
  },
  "numbers": [
    {
      "label": "Clean correct bookings, code decides (F) against the model’s yes counts (E)",
      "value": "41% vs 60% (payment), 14% vs 43% (schedule), 24% vs 50% (doctor)",
      "note": "Gemma 4 26B-A4B. Right, with no dependency left open; that also excludes every self-pay booking, since the driver never asks about a claim or the most the patient will pay. Counting every right booking: 41% vs 63%, 14% vs 51%, 24% vs 58%. Of patients who wanted a booking: 27,840, 22,020 and 23,220 conversations per cell."
    },
    {
      "label": "Largest change in clean correct bookings from swapping the reader",
      "value": "2.9 points",
      "note": "Haiku ahead of Gemma under E on schedule and doctor (paired 95% intervals 2.1 to 3.6), because Gemma’s extra right bookings were lucky guesses. Under F, 1.1 or less. Counting every right booking, 2 points at most, Gemma slightly ahead under E. Single runs: Haiku’s run-to-run variation isn’t in the intervals. The consent rule moved clean correct bookings 19 to 32 (intervals 16.0 to 34.5)."
    },
    {
      "label": "Bookings made on an open time or doctor, model’s yes counts (E)",
      "value": "Gemma: 7 to 13% at two questions, 21 to 27% at three, any pressure. Haiku: under 6% except three questions with urgency alone, 10 to 13%",
      "note": "Schedule and doctor, hedged answers, 1,000 conversations per setting; only breaks on the hedged time or doctor. One question: 0.5% or fewer for both. Never settled: fewer conversations reach any booking, so rates fall. Pooled over every setting and any open dependency: Gemma 11.0% (schedule, 95% interval 9.6 to 12.4) and 13.6% (doctor, 11.9 to 15.3), Haiku at most 3.8%; at the contract’s conditioning nobody crosses the bench’s 1-in-10 line. Exploratory. Under F, none."
    },
    {
      "label": "Conversations with a wrong write, model’s yes counts (E)",
      "value": "Gemma 453, 526, 750; Haiku 400, 213, 236",
      "note": "Payment, schedule, doctor, of 48,000 conversations each. A conversation with a wrong write has at least one booking, move or cancellation the patient’s truth didn’t allow. Haiku with thinking on: 375, 208, 222. Under F: 0 to 10 per cell."
    },
    {
      "label": "Reached a person, of 15,600 who needed one (payment, F)",
      "value": "Gemma 34%, Haiku 22%",
      "note": "Paired gap 10 to 14 points in every arm and prompt (type-resampled intervals −22.8 to −4.6 at the widest). In 1,727 of the 1,914 conversations where Gemma reached a person and Haiku didn’t, Haiku had answered a self-pay question no, mostly to a patient saying they couldn’t decide yet; Gemma answered the same line unclear in 1,686. Haiku thinking on 22%, with worked examples 20%; Gemma with worked examples 34%. Under E: 30%, 20%, 21%, 19%, 30%. Schedule and doctor: readers within 1.1 points."
    },
    {
      "label": "How readers answered a self-pay hedge (payment, F)",
      "value": "Gemma: unclear 3,970, no 0, yes 87. Haiku: unclear 2,520, no 1,216, yes 13",
      "note": "Self-pay questions, not conversations, whose line matched four hedge phrases (“not sure”, “check with”, “come back to that”, “think about”). The filter misses other can’t-decide lines such as “I’d need to check and get back to you”, so it undercounts Haiku’s no answers; the conversation-level count is 1,727. Haiku thinking on: unclear 1,753, no 1,710, yes 21. With worked examples: Haiku unclear 1,611, no 1,853, yes 38; Gemma unclear 3,981, no 11, yes 57."
    },
    {
      "label": "Why the code rule lost clean bookings",
      "value": "96 to 99% were a clear yes longer than four words",
      "note": "Conversations E booked cleanly and F didn’t, every cell. Payment, Gemma: 5,318 of 5,413 were “Yes, that’s right, please go ahead.” (seven words). Schedule, Gemma: 4,906 of 6,716 had urgency added (“Yes, book it. I really need this soon.” is eight words), 1,671 the long form. F asks again and the patient repeats it. F booked cleanly where E didn’t in 15 to 246 per cell. On raw right bookings the share is 86 to 93%."
    },
    {
      "label": "Worked examples, payment",
      "value": "Clean bookings up a little; conversations with a wrong write under E: Gemma 453 to 851, Haiku 400 to 596",
      "note": "Of 48,000 each. Clean correct bookings: Gemma 11,281 to 11,585 (F) and 16,599 to 16,911 (E); Haiku 11,551 to 11,583 and 16,893 to 16,939. Six examples per question from Lab 024, the self-pay hedge relabeled unclear, sent as prior chat turns. Wrong writes rose in list-then-book, unconfirmed booking, change of mind and pivot, and for Gemma in ineligible coverage."
    },
    {
      "label": "Breaks under the code rule (F)",
      "value": "All of them: self-pay bookings made without asking about a claim or the most the patient would pay",
      "note": "40 to 131 per cell, every cell and reader. Every self-pay booking either arm made skipped those questions (Gemma, payment: 125 of 125 under F, 983 of 983 under E). It’s the driver’s gap."
    },
    {
      "label": "Conversations scored",
      "value": "1,056,000",
      "note": "22 cells of 48,000. 1,000 patient truths each over 8 identities (on payment, 580 wanted a booking, 325 needed a person and 95 neither; on schedule and doctor, who needs what shifts with the setting, and 300 conversations count as both). No infrastructure errors."
    },
    {
      "label": "Time per cell",
      "value": "Gemma 364 to 541 minutes; Haiku 68 to 103",
      "note": "Gemma on one DGX Spark, 4 conversations at a time; Haiku through Anthropic’s API, 16 at a time. Lower ends are the few-shot cells. Two deployments, not a model speed test."
    },
    {
      "label": "Metered spend",
      "value": "$58.65",
      "note": "All Haiku runs, 14 cells plus slices and canaries, at list price ($0.10 and $0.50 per million tokens; cached input $0.01). Matched Anthropic’s console to the cent. Gemma: $0 metered."
    }
  ],
  "opened": [
    {
      "q": "What happens when the model runs the whole workflow, not just the reading?",
      "a": "That’s the next lab. It puts Lab 23’s agent kit, where the model decides every step and writes every reply, on the same simulated patients. Because the kit writes its own replies, the patient needs a model to hear it, and that hearing has to be calibrated before the results count."
    },
    {
      "q": "Should the code rule accept a clear yes of any length?",
      "a": "Of the conversations F lost cleanly, 96 to 99% in every cell were a clear yes over four words: the polite long form, or a yes with urgency added. A rule that accepts a clear yes of any length targets most of them, but it hasn’t been run; whether it would also admit a hedge is the measurement to make before changing it. So is a patient who shortens a yes when asked again."
    },
    {
      "q": "Why does Haiku answer a can’t-decide-yet patient no, and why does urgency alone move it?",
      "a": "Thinking didn’t change the hand-off gap and worked examples made it worse. Whether an explicit definition of unclear in the driver’s question changes it, without making Haiku book more on a pressed hedge, is untested."
    }
  ],
  "not_proved": [
    "These are behaviors inside a simulation. The patients are code: they speak in templates and hear the agent through a deterministic parser. The lab shows how the parts compare here, not production rates, and not that the ordering holds with real callers.",
    "One clinic: 8 patient identities, 6 doctors, 42 slots, Lab 23’s 18 types. The 1,000 patients are generated truths over those 8 identities.",
    "Each cell’s 48,000 conversations are 1,000 patients at 48 settings. A patient’s truth is fixed across its 48 (on schedule and doctor, a setting that never draws the answer out changes the right outcome to a person, not the truth), so the 1,000 patients are the independent observations; every interval resamples whole patients.",
    "The contract Lab 029’s runs were frozen under names a logistic model with a patient random effect for the thresholds, one dial at a time with the others at their easiest level, plus a pilot, a combined pass and a fallback sensitivity. None of those ran. The rates here are raw shares with patient-bootstrap intervals, pooled over the other two dials, plus raw rates at the contract’s conditioning (other dials at their easiest), where no reader crosses 1 in 10. That deviation is logged in this record and the results write-up, not in the frozen change log, because an entry there would change the hash every cell carries. Treat the threshold lines as exploratory; the rule-against-reader comparisons are paired differences on the frozen set.",
    "The driver’s prompts were written for Gemma in Lab 23 and weren’t tuned for Haiku. A prompt written for Haiku might read hedges differently.",
    "The patients come from Lab 23’s set (same 8 identities, 18 types at the same counts) but speak templates written during the build, not Lab 23’s model-written lines, and they don’t rephrase. The instrument is more controlled than Lab 23’s; its patients talk less like people.",
    "Intervals resample patients. Resampling the 18 types instead widens them (rule gap on clean bookings 13.8 to 38.9 points, reader gap inside 4.2) without changing the ordering. The 8 identities weren’t resampled.",
    "Haiku has no seed, and each Haiku cell ran once. Two runs of the same 576-conversation slice differed in 1 outcome, but a full cell wasn’t repeated, so the reader bounds are for these runs and leave out Haiku’s run-to-run variation. Gemma on vLLM repeated word for word across two Sparks and two runs.",
    "Worked examples ran on payment only. Schedule and doctor with examples weren’t run.",
    "The bench’s “right” outcome doesn’t require a booking’s dependencies to be settled: a booking can be right and booked too soon, a lucky guess. The headline counts clean correct bookings (right, no dependency open), which also excludes every self-pay booking because of the driver’s claim gap; the raw count and a count excluding only that gap are given beside it. The bench’s contract asks for writes per conversation beside break rates; the hedged rates here were checked against them but the record reports them only where they change a reading.",
    "Gemma with reasoning on, Qwen, a model running the whole workflow and a model-written patient voice weren’t run on the entire set. A model running the whole workflow is the next lab.",
    "Haiku’s costs are list price from reported tokens, matched to the console. Gemma’s $0 is metered cost only; the Sparks’ purchase, power and my time aren’t in it."
  ],
  "executive_summary": [
    "Lab 23 compared two ways of building a clinic booking agent and found both leaked wherever a check trusted the model’s word. Its code-driven version (DCITL) had two consent rules. Arm E let the model’s yes count. Arm F decided consent only in code, from an explicit “I confirm” or a bare yes of four words or fewer. In Lab 23’s pressure tests F made no forbidden booking, though two driver bugs wrote unconsented changes in about 1 in 1,000 sessions. Its scripted patients were told to confirm explicitly, so it couldn’t say what the rule costs when patients don’t.",
    "This lab asks that question on a different instrument, and a second one: does the reader model matter? clinicbench’s patients are code. Each one’s truth is set before the call, and code decides what the patient says, how clearly, and how they respond to what they’re asked. Every booking is scored against the truth. A booking made while something it depends on was still unclear (the time, the doctor, whether the patient will pay) counts as booked too soon; the bench’s threshold is 1 in 10 conversations. The patients come from Lab 23’s set, but they react where Lab 23’s were scripted and speak templates where Lab 23’s spoke model-written lines, so the two labs’ numbers don’t compare.",
    "Each cell is 1,000 patient truths over 8 identities and Lab 23’s 18 types, each heard at all 48 settings of one dimension: 4 levels of how clearly they state it, 4 of how many questions it takes, 3 of pressure. That’s 48,000 conversations. On payment, 580 patients (27,840 conversations) wanted a booking and 325 (15,600) needed a person; on schedule and doctor, a setting that never draws the answer out turns some bookings into hand-offs, so the counts are 22,020 and 21,720 (schedule) and 23,220 and 20,520 (doctor), with 300 conversations in both. I ran F and E on all three dimensions with three readers: Gemma 4 26B-A4B on owned DGX Sparks, and Claude Haiku 5.5 through Anthropic’s API with thinking off and on. Then F and E on payment with six worked examples per question, on both readers. 22 cells, 1,056,000 conversations, no infrastructure errors.",
    "The consent rule dominated correct bookings. Counting only clean ones, right with no dependency left open, E booked correctly 19 to 32 points more often than F with every reader (paired 95% intervals 16.0 to 34.5; 13.8 to 38.9 when resampling the 18 types instead of patients). Counting every right booking, 21 to 37. Clean also excludes every self-pay booking, because the driver never asks about a claim or the most the patient will pay; excluding only those, the gap is 20.5 to 34.2. Swapping the reader moved clean correct bookings 3 points at most: under F, 1.1 or less; under E, Haiku ahead by 2.7 to 2.9 on schedule and doctor (intervals 2.1 to 3.6), because Gemma’s extra right bookings were lucky guesses (schedule: 1,646 of Gemma’s 11,183 right bookings had a dependency open, against 771 of Haiku’s 10,951). These are single runs; Haiku’s run-to-run variation isn’t in the intervals. Of the conversations E booked cleanly and F didn’t, 96 to 99% were clear yeses longer than four words that the patient repeated when asked again.",
    "Under E, the readers failed on different settings. Counting bookings made while the hedged time or doctor itself was still open: with one question, 0.5% or fewer for both readers. Gemma rose with every extra question at every pressure level, 7 to 13% at two and 21 to 27% at three. Haiku stayed under 6% in 11 of the 12 settings of questions and pressure, and reached 10 to 13% in one, three questions with urgency alone. Where the patient could never settle it, fewer conversations reached any booking (5 to 27% did, against 22 to 46% at three questions), so those rates fell. Pooled over every setting and counting any open dependency, Gemma’s hedged rate was 11.0% on schedule (95% interval 9.6 to 12.4) and 13.6% on doctor (11.9 to 15.3), past the bench’s 1-in-10 line on that pooled rate; at the contract’s conditioning (one question, no pressure) nobody crosses it. Treat the line as exploratory. Under F no booking was made on an open time or doctor. Every break under F, in every cell, was the driver’s: a self-pay booking made without asking about a claim or the most the patient would pay (40 to 131 per cell). Every self-pay booking either arm made skipped those questions.",
    "Haiku sent fewer patients to a person, under either rule. Asked whether to proceed self-pay, some patients say they can’t decide on this call (“I’d have to check with my husband”, “I’d need to check and get back to you”). Gemma answers those unclear and the driver keeps asking; Haiku answers more of them no and the driver ends the call. On payment under F, Gemma reached a person and Haiku didn’t in 1,914 paired conversations and the reverse in 52; in 1,727 of the 1,914 Haiku had answered a self-pay question no, and Gemma had answered the same line unclear in 1,686. Under E, 1,438 of 1,696. Reached a person on payment: Gemma 34%, Haiku 22%, a paired gap of 10 to 14 points in every arm and prompt. On schedule and doctor the readers were within 1.1 points. Most patients who needed a person missed one under either reader: under Gemma on payment, 7,975 of 15,600 were never handed off and 2,317 gave up.",
    "Cost and time per cell: Gemma 4 26B-A4B ran on owned hardware at $0 metered in 442 to 541 minutes (364 for the few-shot cells), four conversations at a time on one DGX Spark. Haiku cost $3.00 to $5.60 a cell at list price and ran in 70 to 103 minutes (68 for the few-shot pair, side by side), sixteen at a time. The venues and parallelism differ, so the times compare deployments, not models. All of Haiku’s runs came to $58.65, which matched Anthropic’s console to the cent."
  ],
  "dapm": [
    {
      "instrument": "4plus1",
      "reading": "decision",
      "layer": "layer3",
      "placement": "Retained",
      "cells": {
        "f": "Retained",
        "e": "Retained"
      },
      "note": "Who decides the next step in the scheduling workflow. In both arms code runs the workflow and the model only interprets what code can’t read."
    },
    {
      "instrument": "4plus1",
      "reading": "decision",
      "layer": "layer2b",
      "placement": "Retained",
      "cells": {
        "f": "Retained",
        "e": "Retained"
      },
      "note": "Who decides, at execution, whether a write goes through. The clinic’s tools check slots and records themselves in both arms; neither takes a fact from the model."
    },
    {
      "instrument": "4plus1",
      "reading": "decision",
      "layer": "layer2c",
      "placement": "Retained",
      "cells": {
        "f": "Retained",
        "e": "Delegated"
      },
      "note": "Who determines that the patient agreed, which here is what “done” means for a booking. Under F, code ignores a model’s yes; only the patient’s own explicit words commit. A model’s no still routes the call: the driver ends it, which is how the reader moves hand-offs under F. Under E, code commits on the model’s yes to a closed question: Delegated within that question. The reader model doesn’t change this placement; it changes how well a Delegated decision goes, and where a no sends the patient."
    }
  ],
  "seam_map": null,
  "writeup": [
    "Lab 23 left a question its scripts couldn’t answer. Its code rule made no forbidden booking in its pressure tests, but its patients were told to confirm explicitly and couldn’t react to the agent. What does the rule cost when patients talk back? Answering that took a different instrument, with patients who react and whose words came from a different writer, so this lab’s numbers stand on their own. They sit next to Lab 23’s; they don’t correct them. And a second question: if you let a model decide consent, does it matter which model?",
    "Answering that needed a patient I controlled completely. clinicbench’s patients are code. Before the call, each one’s truth is fixed: the most they’ll pay, the times that work, the doctor they want. A setting decides how clearly they say each thing: plainly, hedged, contradicting themselves, or not at all; how many questions it takes; and whether they push. Code logs what they disclosed and how clearly, and every booking is scored against the truth at that moment. No model grades anything. I froze the instrument by hash and fixed three persona defects in a shakedown before any cell ran.",
    "The driver is Lab 23’s, unchanged. Arm F decides consent in code: only an explicit “I confirm” or a bare yes of four words or fewer commits. Arm E lets the model’s yes count. I ran both over the entire set on all three dimensions with Gemma 4 26B-A4B on owned DGX Sparks, then swapped in Claude Haiku 5.5 through Anthropic’s API with thinking off and on.",
    "The rule mattered most for bookings. Counting clean bookings only, right with no dependency left open, E booked correctly 19 to 32 points more often than F, with every reader. The reason is mundane. A patient who talks at length says “Yes, that’s right, please go ahead.” That’s seven words. One pressed on the time or the doctor says “Yes, book it. I really need this soon.” That’s eight. F asks again for a plain yes or no, gets the same answer, and hands off. This bench’s patient repeats its answer when asked again; a real caller might shorten it. Letting the model decide bought those bookings with wrong writes: hundreds of conversations per cell that F didn’t have.",
    "The reader mattered for safety once it was allowed to decide, and the two readers failed on different settings. When one question settled a hedge, neither booked on the open time or doctor more than 0.5% of the time. With two or three questions, Gemma did in 7 to 27% whatever the pressure. Haiku stayed under 6% except in one setting, three questions with urgency alone, where it reached 10 to 13%. On clean correct bookings that made Haiku 3 points better than Gemma on schedule and doctor; Gemma’s extra right bookings were lucky guesses. If your design lets the model’s yes count, test the reader on slow hedges and on pressed ones.",
    "Haiku lost patients on payment, and the driver did the rest. Asked whether to pay out of pocket, some patients say they can’t decide on this call: “I’d have to check with my husband.” Gemma answered those unclear, and the driver kept asking and sometimes found them a person. Haiku answered more of them no, and the driver ended the call. In 1,727 of the 1,914 conversations where Gemma got a patient to a person and Haiku didn’t, Haiku had said no to a self-pay question, and Gemma had said unclear to the same line in 1,686. A no isn’t an unreasonable reading of “let me check with my husband”, but the bench defines it as unclear, and the driver treats no as final. Thinking on answered more of those no and left the hand-off rate at 22%. Lab 024’s worked examples, four of five self-pay examples labeled no, moved Haiku further toward no and left Gemma alone; they added a few hundred clean bookings and raised conversations with a wrong write under E on both readers, Gemma 453 to 851 and Haiku 400 to 596. And most payment patients who needed a person missed one under either reader.",
    "The cost is a different trade than I expected. Haiku costs about $3 to $5.60 a cell and ran one in about 80 minutes at sixteen conversations at a time through Anthropic’s API. Gemma costs nothing per call on hardware I own and took 6 to 9 hours at four at a time on one Spark. Those are two deployments, not a model speed test. At this scale the marginal price is small either way, and the Sparks’ purchase isn’t in Gemma’s $0. What separates them is time, and whether you can tune and keep the weights."
  ],
  "abstraction": [],
  "method": [
    "Self-funded. The DGX Sparks and the Mac Studio were bought by the practice. No vendor was briefed, reviewed this, or was given advance sight.",
    "Instrument: clinicbench v1, built and run by Claude Code (an Anthropic tool) under the author’s design rulings, frozen by hash before the cells (freeze sha256 prefix 15e801051e5abc3f). Lab 23’s driver was vendored unchanged from its frozen original. The clinic itself traces to the clinic appointment example in Jeremy Schulman’s Promise-Driven Agentic Kit, which Lab 23 started from and expanded; clinicbench ports Lab 23’s expanded world. The measurement contract was frozen before the build and changes before the freeze are logged with reasons; the analysis deviation after the runs is logged in this record (see what this doesn’t prove). A shakedown on Gemma found three persona defects, all fixed and rerun before the freeze. Paired contrasts, clean correct bookings, the breakdown of F’s losses, the hand-off decomposition, the patient lines behind it, and proposition-specific hedged rates by probes and pressure were computed from the built cells after review.",
    "Venues: Gemma 4 26B-A4B (bf16, vLLM 0.25.1, revision 20da991a pinned) on two DGX Sparks, one cell per node; a parity check gave word-for-word identical transcripts on both. Claude Haiku 5.5 through Anthropic’s API via LiteLLM 1.106.2, thinking off or adaptive at medium effort, checked against a local stand-in before any spend. Worked examples: Lab 024’s final set, one self-pay example relabeled, sent as prior chat turns with the driver’s prompt unchanged.",
    "Reviewed before publication by two models from other families (ChatGPT and Gemini, four rounds); their points on the code rule’s confound, lucky-guess bookings, denominators, intervals, conditioning and routing are reflected in the record, several through analyses run after review.",
    "The bench is clinicbench, published open under the MIT license at github.com/kltownsend/clinicbench: the patients, the scorer, the runner and its gates, the code-driven driver with both consent rules, and a public contract with its own freeze. Lab 029’s private freeze stays with this lab. The method for assessing a vendor’s placement is not published."
  ],
  "assessments": [],
  "publisher": "The Advisor Bench LLC",
  "program": "Layer2C Labs",
  "url": "https://labs.layer2c.com/labs/who-decides-consent",
  "markdown_url": "https://labs.layer2c.com/labs/who-decides-consent.md",
  "json_url": "https://labs.layer2c.com/labs/who-decides-consent.json"
}