# Lab 015 raw lab detail: Inherit the boundary. Own the gate. Working notes behind the published entry. Every substrate detail ships here: the API mechanics, the audit tooling, the numbers, and the measurement errors that were caught and corrected on the way. **What stays back.** Two things, for different reasons. The calibrated assessment methodology (the ratified grading rules, worked reference rows, thresholds, axis weighting) is the practice's proprietary work and never appears in a lab download. And the synthetic instrument's prompt text is withheld deliberately, because the platform under test is administered by its vendor and an instrument the administrator can read is not an instrument under test. What ships is the harness that ran it and the deterministic tooling that audited it. Returns, not algorithms. **Environment identifiers are redacted.** The deployment is the sponsor's demo environment. Workroom, deployment and agent UUIDs are replaced with labels. --- ## The one question A bench of analysts running one shared assessment instrument is only credible if the evidence base stays under central control. The pitch: a curator vets a corpus per workroom, analysts consume it. The loss condition: an analyst reaches into another analyst's vendor, or scores against evidence nobody vetted, while every boundary in the product reports success. ## The apparatus Kamiwaza core 1.0.1, `authMode: rebac`, environment `production`, on the vendor's demo deployment. Two workrooms, one vendor corpus each, eight public press releases per corpus. Dell 76,213 characters, Supermicro 52,967. Corpora built by a fetcher that records per-document URL, SHA-256, character count, extraction container and retrieval timestamp; the manifests are committed, the document bodies are not (they are the vendors' published pages). One Kaizen agent per workroom. Kaizen is Kamiwaza's agent surface, built on OpenHands. Model pinned to `Qwen3-VL-30B-A3B-Instruct` served by vLLM on the same deployment, temperature 0. Three principals: a curator holding an administrative personal access token, and one analyst per workroom holding `editor` on that workroom only. Nine assessment runs. Six same-vendor (three per workroom), two reciprocal (each workroom asked about the other's vendor), one null probe (a third vendor loaded nowhere). ## The measured spine | measurement | result | | --- | --- | | Grounded claims, six same-vendor runs | 42 of 42 resolve to the asking workroom's own corpus, verbatim | | Cross-workroom citations | 0 | | Retrieval hits under 51 adversarial cross-vendor searches | 241, none from the other workroom | | Reciprocal probe layer cells | 32 of 32 returned insufficient evidence | | Invented capability claims across 75 completed cells | 0 | | Ingest routes closed to a `viewer` | 4 of 4 | | Roles separating "run the instrument" from "write the corpus" | 0 | | Corpus restore, 8 documents | ~4 minutes per workroom | Reciprocal detail: asked 25 times about the other vendor, the Dell workroom returned 120 hits, all Dell. Asked 26 times, the Supermicro workroom returned 121 hits, all Supermicro. Same-vendor per run: Dell 8/8 layers on all three runs (12, 9, 11 claims). Supermicro 7/8, 7/8, 6/8 (6, 2, 2 claims). Layer 3 never completed on either corpus; the agent spent 23 to 30 searches hunting application-layer material that eight hardware press releases do not contain. ## Write control, as probed Attempted as a `viewer`, each verified by reading state back rather than by status code: | route | result | | --- | --- | | `POST /context/upload/` | refused, names the required relation | | collection create | refused | | vector database create | refused | | direct vector insert (bypasses the document pipeline) | refused, at a stricter relation | | cross-workroom read and write | refused | | agent create (UI and API) | `403 "Viewer access is read-only in this workroom."` | | start a conversation | refused, so no tool is reachable | Read-scoped and write-scoped personal access tokens produced identical outcomes on every one. **Token scope is not an authorization boundary on this platform.** The workroom role does the work. ## Model behavior, in its own column At temperature 0, across three Dell runs: seven of eight layers identical, one (Layer 0) flipped to a full refusal in one run. Claim counts drift inside stable grades. The null probe returned insufficient evidence on every cell for a vendor loaded nowhere, so the parametric floor is zero and the reciprocal refusals cannot be explained as indiscriminate refusal. Ten of 75 completed cells carry a `gap` rating with no citation. Every one is an assertion of absence; none is a positive capability claim. The instrument's own vocabulary is ambiguous between `gap` (the vendor has little here) and insufficient evidence (this corpus cannot say), and the model chose the same one consistently in the same cells. That is an instrument wording issue, not a platform result. ## The audit tooling Deterministic Code In The Loop applied to the lab's own acceptance bar. Four questions are mechanical and the resolver decides them alone: - does the cited document exist in that analyst's corpus - does the quoted span appear verbatim in it - does the citation resolve into the *other* analyst's corpus (a retrieval leak, mechanical either way) - is the claim uncited One question escalates: does the resolved span actually support the claim. Across nine runs the mechanical gate handled everything but the judgement call. Verdicts: `ESCALATE`, `QUOTE_NOT_VERBATIM`, `FABRICATED_QUOTE`, `UNRESOLVABLE`, `NO_QUOTE`, `UNCITED`, `RETRIEVAL_LEAK`, `CROSS_CITE_UNVERIFIED`. **Corpus ownership is decided by the workroom, never by the vendor asked about.** The reciprocal probe asks workroom A about vendor B; keying on the vendor would label B's corpus "own" for an A run, and a genuine cross-workroom leak would score as a clean citation. The corpus validator is the compensating control a customer has to build. It enumerates every collection in the workroom through the vector store and diffs against the vetted manifest, in both directions (unvetted present, vetted absent). Enumeration failure returns `UNKNOWN`, never `clean`. ## Gotchas that cost hours **Authentication.** Personal access tokens work on the platform API and return `401` at any application's session layer. Use an OAuth access token from `POST /auth/token`. **Workroom binding.** Passing `workroom_id` on a call is not enough for writes. Bind the session first, or you get `403 "Authenticated workroom context missing."` — which is a *different* refusal from a permission denial and must not be read as one. Two apparent findings in this lab were this error. **The agent's model config.** Three things must combine and none works alone: the model name is provider-prefixed, the provider is the platform, and the endpoint is a path with the base URL left null. Learned by reading a UI-created agent after eight failed guesses. **Agent identity must key on a hash of the whole config.** An agent carries its configuration immutably, so reusing a name after any change silently runs the old one. This bit twice and both times looked like the fix had failed. **Do not use the SDK's `chat()`.** It treats an agent error event as terminal while the agent recovers and completes. Drive with send-message, then run, then poll events. **The answer arrives in a finish action, not a message event.** An extractor reading only message events reports a completed layer as incomplete. That cost a full eight-layer run and made a four-minute session ceiling look like a hard blocker. **Poll cadence dominated wall-clock.** Waiting for three stable five-second polls before re-driving cost ~15 seconds on a ~6-second turn. Polling at 2s and re-driving after one stable poll took an eight-layer pass from "does not fit" to 297 seconds. **`/context/upload/` needs its trailing slash.** Without it the server returns `307`, and `urllib` does not re-POST a body across a redirect, so the file goes nowhere while the client reads a status and moves on. The multipart field is `file`, singular, one file per request. **Collection naming is asymmetric.** `GET /context/collections/` returns `ws__default`. The write path applies that prefix itself, so passing back the name you were just given creates a *second* collection, `ws__ws__default`. The load reported `201`, `completed`, eight of eight successful, 193 chunks — into a collection no agent reads. Pass the bare name. **Never read `colls[0]`.** List order is not stable. With a stray collection present, index 0 decides what you see, and the loader and the validator reported confidently different truths about the same deployment seconds apart. **Deleting a collection does not clear source tracking.** Dedup keys on source URN plus content hash. Re-uploading previously-ingested documents through the upload endpoint returns `201`, completes, and records `status: "skipped"`, `chunks_created: 0`, `error: null`, `terminal_outcome: "indexed"` — with zero vectors written. The upload endpoint exposes no override; the documented `force` flag lives only on the pipeline-job endpoint, which takes inline content. **No per-document delete exists.** Removing one contaminating file means deleting the collection and force-reloading the vetted set. **The activity feed is a 100-entry rolling window.** Measured at 13m36s of history when idle and **2m23s** while six assessment runs executed. It is bounded by entry count, not time, so the busier the deployment the less history survives. ## Status does not reflect reality The through-line, across subsystems, and the cause of every retraction in the working log: - `201` on an upload that never indexed - `200` on an agent create that never created - `DEPLOYED` on a model whose upstream returned `502` - `terminal_outcome: "indexed"` on a document with zero vectors - a complete, successful, eight-of-eight load into a collection the agent cannot read Every compensating control in this lab enumerates state instead of reading a status field. Each time that discipline lapsed it produced a wrong answer. ## Errors caught and corrected Recorded because the corrections are the method, and because several would have published something false. **Five "the platform cannot" claims dissolved** on one more question: a forward-auth sweep, tool restriction, corpus enumeration, an apparent erased audit trail, and a material-defect claim about re-ingestion that was drafted for vendor notice before a fresh upload disproved it in ten seconds. The rule that came out of it: before any defect claim, read the subsystem's documentation end to end. Grepping it produced four of the five. **The resolver was wrong twice, both times against the model.** It reported a fabricated quote where the model had wrapped a 299-character verbatim span in single quotes while the document used double quotes, and again where the model wrote "first to market" against the document's "first-to-market" with the other 480 characters identical. Under the instrument's own rules a fabricated quote is a finding about the *model*, so a resolver bug was one step from becoming a published accusation that the model invented a vendor executive's quote. Enclosing delimiters and hyphenation are now normalised, and `QUOTE_NOT_VERBATIM` separates a mistyped span from an invented one. **The near-verbatim check was wrong on its first implementation.** Eight-word shingles meant one mistyped word in a 40-word span poisoned eight of thirty-three shingles and scored 0.76 — indistinguishable from invention. Word-level sequence matching costs one word per error instead of eight. **The validator was wrong twice, both times toward `clean`.** An early version sampled by search and reported clean on a corpus contaminated ten minutes earlier; sampling is not enumeration. A later version read `colls[0]` and missed a corpus sitting in a stray collection. **Four wrong checks in total, every one failing toward a clean-looking answer.** The common cause is writing a check in the same sitting as the thing it checks, never adversarially. The self-test is the control, and its fixtures are now generated from the corpus at import time rather than written from memory — the hand-written version asserted text that was not in the file and failed as a test bug. ## What this does not establish Nothing here is a security result. Every account was authorized and acting in good faith inside its own workroom; no privilege escalation or credential attack was attempted. One release, one deployment, two workrooms, one workload. The isolation result is one run per direction across 241 retrieval hits, not a repeated trial with a confidence interval. The cross-vendor questions were the instrument's own searches, not a probe set with known documented answers frozen before the workrooms opened. That measures conflation. It does not deliver the sharper leak-candidate signal a frozen probe set would. Whether a curator can read another user's retrieval trail was not established: reading it requires an application session, a personal access token is refused at that layer, and the bench held no owner-role password. That is a limit of the bench's access, not a property of the platform.