# Lab 022 raw lab detail: Memory bandwidth sets how fast a 31B dense model works, not how well.

One Apple Mac Studio, M5 Ultra (36-core CPU, 80-core GPU), 256 GB of unified memory, 4 TB, bought
by the practice. Compared with one NVIDIA DGX Spark (GB10, 128 GB) and with two Sparks clustered
over one 200 Gb/sec RoCE link, both bought at retail. The Studio runs were made on
October 18 and 19, 2026. The single-Spark arm ran on August 3 and 4, 2026, and the clustered arm
in August on Lab 014's cluster. Neither Spark arm was published before this lab. Lab 013 published
that the 31B can't practically run this harness on a Spark; these arms are the evidence behind that. Nothing was rented. Metered spend: $0.

This download carries the substrate: configurations, pins, per-task results, the failures and how
they were found. The assessment methodology, the vendor grades and the judgment that turns a score
into a placement are not published.

## The one question

What does memory bandwidth do to a model with many active parameters doing agentic work, when the
model fits on both boxes? Decode speed follows the bytes read per token: active parameters times
precision. The test model is Gemma 4 31B, a dense model, so all 31 billion weights are active. A
small dense model or a mixture-of-experts model with few active parameters reads far less and runs much
faster, though below its bandwidth ceiling: Lab 016 measured a 12B dense model at 4 bits at 22.0
tok/s on the same Spark and Gemma 4 26B-A4B (about 4B active, NVFP4) at 30.0. At 4 bits the 31B
model is about 15.5 to 18 GB depending on the file, so capacity is not the constraint on either box.

Published bandwidth: 273 GB/s for the GB10, 1.2 TB/s for the M5 Ultra (4.4x). Neither was measured
here.

## The instrument

The same instrument as Labs 011 to 014, unchanged. Twenty-two Python repair tasks reconstructed from
maintainer-merged pull requests, each with held-out tests the model never sees. A deterministic gate
applies the model's edit, runs the tests in a network-isolated container, and rules. The harness is
a headless Claude Code session (v1 harness, Claude Code 2.1.218) with file read, edit and shell,
reaching the model through a litellm proxy. Two sessions at a time, 7,200-second ceiling per task,
hardest task first.

The harness ran on a DGX Spark for every arm. For the Studio arms it reached the Studio's server
through an SSH relay, because bench code is not run on a machine on the author's personal network.

## The measured spine

| | One DGX Spark | Mac Studio | Two Sparks, clustered |
| --- | --- | --- | --- |
| Runtime | vLLM (NVIDIA image), Aug 2026 | llama.cpp b11399 | vLLM 0.25.2.dev0 (community image) |
| Model file | nvidia/Gemma-4-31B-IT-NVFP4 | unsloth Q4_K_M | nvidia/Gemma-4-31B-IT-NVFP4 |
| Single-stream decode, 256-token probe | 7.0 tok/s | 42.8 tok/s | not measured on this probe |
| Sessions | 2 | 2 | 4 |
| Tasks run | 11 | 22 | 22 |

The two-Spark column is context, not part of the ruling: it ran four sessions to the others' two, on a different image and proxy route, and its sessions repeated tool calls far more often. Its per-turn time was faster than one Spark's and it took nearly three times the turns.

Same 11 tasks (the ones the single Spark ran, which Lab 013's rented L40S venue had failed). Timeouts count at the 120-minute ceiling and as misses, because ceiling truncation is part of what this lab
measures. Labs 012 and 013 later scored a ceiling-interrupted session as no verdict; under that convention
the single Spark reads 5 of 10. Excluding the one-Spark timeout, its median is 59.1 min:

| | One Spark | Studio | Two Sparks |
| --- | --- | --- | --- |
| Solved | 5 | 6 | 0 |
| Timeouts | 1 | 0 | 1 |
| Median per task | 61.0 min | 12.8 min | 78.3 min |
| Summed task time | 10.86 h | 3.79 h | 13.46 h |
| Turns | 126 | 176 | 363 |
| Seconds per turn | 310 | 78 | 133 |

Full pool:

| | Studio (run 2) | Two Sparks (August arm, Lab 014's cluster) |
| --- | --- | --- |
| Solved | 15 of 22 | 4 of 22 |
| Timeouts | 0 | 3 |
| Session errors | 0 | 3 |
| Median per task | 7.4 min | 75.9 min |
| Summed task time | 4.76 h | 25.06 h |
| Elapsed | 2 h 30 min | not comparable (four sessions) |

## Per task, the matched 11

| Task | One Spark | Studio | Two Sparks |
| --- | --- | --- | --- |
| click-pr3678 | miss, 68.0 min | miss, 19.2 min | miss, 53.4 min |
| more-itertools-pr1117 | solved, 66.2 min | miss, 79.0 min | miss, 10.3 min |
| more-itertools-pr944 | miss, 26.9 min | solved, 7.7 min | timeout, 120.0 min |
| prettytable-pr289 | miss, 61.0 min | miss, 38.6 min | miss, 80.6 min |
| prettytable-pr296 | miss, 90.9 min | miss, 36.9 min | miss, 74.6 min |
| prettytable-pr299 | timeout, 120.0 min | miss, 12.8 min | miss, 59.2 min |
| prettytable-pr332 | solved, 57.1 min | solved, 12.9 min | miss, 84.7 min |
| prettytable-pr344 | solved, 65.8 min | solved, 3.4 min | miss, 60.4 min |
| requests-pr7308 | solved, 44.0 min | solved, 2.7 min | miss, 78.4 min |
| requests-pr7502 | solved, 43.4 min | solved, 8.7 min | miss, 108.0 min |
| task-r029 | miss, 8.0 min | solved, 5.8 min | miss, 78.3 min |

Three tasks flip between the single Spark and the Studio: more-itertools-pr1117 toward the Spark, more-itertools-pr944 and task-r029 toward the Studio. prettytable-pr299 is unsolved on both (a timeout against a miss). This pool flips
borderline tasks between identical runs; a one-task difference is noise.

The parity depends on the ceiling. The one-Spark solves took 43.4, 44.0, 57.1, 65.8 and 66.2
minutes. Under a 60-minute ceiling it keeps three; under 40 minutes, none. The Studio's six solves
took 2.7 to 12.9 minutes. This is a recomputation from the recorded task times, not a run at a
shorter ceiling.

## Studio full pool, run 2, per task

prettytable-pr299 miss 12.8m · prettytable-pr332 solved 12.9m · prettytable-pr289 miss 38.6m ·
prettytable-pr296 miss 36.9m · task-r029 solved 5.8m · click-pr3678 miss 19.2m · click-pr3482 solved
7.2m · dateutil-pr700 solved 7.5m · more-itertools-pr1117 miss 79.0m · requests-pr7502 solved 8.7m ·
prettytable-pr344 solved 3.4m · prettytable-pr345 solved 4.2m · requests-pr7308 solved 2.7m ·
requests-pr7433 miss 15.0m · more-itertools-pr944 solved 7.7m · click-pr3126 solved 1.2m · task-r007
solved 5.9m · more-itertools-pr1119 miss 8.2m · click-pr3504 solved 1.4m · requests-pr7309 solved 1.7m
· prettytable-pr445 solved 4.1m · requests-pr7205 solved 1.7m.

Every task the clustered pair solved, the Studio solved too. The Studio solved all three of the
pair's timeouts. Eight repeated file reads across all 22 sessions.

## Why the decode gap is about 6x, and what it is made of

Decode on a dense model is bounded by bandwidth divided by bytes read per token.

- GB10 ceiling: 273 GB/s over about 15.5 GB per token for the NVFP4 file = 17.6 tok/s.
- Measured on the GB10 (Lab 013): 7.0 tok/s, about 40% of that ceiling by this arithmetic. SM121 has
  no native 4-bit floating-point conversion. An FP8 file ran at 6.8 to 6.9 tok/s, about 77% of its
  8.8 ceiling, because it reads about 31 GB per token. The percentages and ceilings are computed
  here from the published 273 GB/s; Lab 013 published the measured tok/s.
- M5 Ultra ceiling: 1.2 TB/s over about 18 GB = about 66 tok/s. Measured: 42.8 (llama.cpp) and 46.2
  (oMLX), about two thirds of the listed bandwidth by the same arithmetic.

So the measured 6x is part kernel and part memory. Even at its 4-bit ceiling the GB10 would sit 2.4x
under the Studio's measured decode. The floor is conservative: it grants the GB10 100% of its
bandwidth and the Studio only what it achieved. A same-runtime probe on the GB10 (llama.cpp, the same file) was
considered and not run: peer testing of the same pairing reports consistent results.

Studio, further gates:

| Gate | Result |
| --- | --- |
| Decode, 1,024 tokens, 1 stream (oMLX) | 44.9 tok/s |
| Decode, 1,024 tokens, 2 streams (oMLX) | 43.1 per stream, 85.0 aggregate |
| Decode, 1,024 tokens, 2 streams (llama.cpp) | 29.3 per stream, 58.0 aggregate |
| Cold prefill, 35.8K tokens (oMLX, salted) | 1,115 tok/s, 32.1 s to first token, then 25.8 tok/s |
| Cold prefill, 48.1K tokens (oMLX, salted) | 995 tok/s, 48.3 s to first token, then 21.9 tok/s |
| Prefill (llama.cpp) | 909 tok/s at 2.4K, 728 at 38.4K |
| Planted-fact recall, 2.4K to 38.4K, temp 0 and 1.0 | correct on both runtimes |
| Tool-call round trip | pass on both runtimes |

Decode at harness-sized context is about half the short-prompt rate.

## Serving envelopes

| Setting | One Spark | Studio run 2 | Two Sparks |
| --- | --- | --- | --- |
| Thinking | off by default (no reasoning options in the pod definition) | off (`--reasoning off`) | off (no reasoning parser) |
| Sampling | server defaults from the model's generation config | temp 1.0, top_p 0.95, top_k 64, min_p 0 | server defaults from generation config |
| Context per session | 131,072 | 131,072 (two slots of 262,144 total) | 131,072 |
| Memory setting | utilization 0.70, 52 GiB KV | Metal, full offload | utilization 0.60, 53.4 GiB KV (from the arm's recorded venue) |
| Proxy route | litellm `hosted_vllm/` (chat completions) | litellm `openai/` (Responses API) | litellm `openai/` |
| Prefix cache | vLLM default | llama.cpp slot cache, default | on |

The single Spark's exact serve command for that arm was not preserved beyond memory setting, context
and model file; the thinking setting is inferred from the pod definition used for this model.

Pins, Studio run 2: llama.cpp commit 2ca15f5404760548c39e7b92bd43116a09414a1a (tag b11399,
2026-10-04), built from source with Metal; `unsloth/gemma-4-31B-it-GGUF`
`gemma-4-31B-it-Q4_K_M.gguf`, revision c1ac76e99d5513b141e8adde7288b85c3f9c32ec, 18,323,733,440
bytes, SHA-256 38bd64c852c4b460434cc7162fa9bdcf242faf86502581a754cb72956bb17f84. Serve flags:
`-c 262144 -np 2 -ngl 99 --flash-attn on --jinja --reasoning off --temp 1.0 --top-p 0.95 --top-k 64
--min-p 0.0`. The loader warns that three control-looking tokens were not marked as control tokens
and overrides them.

## The oMLX failure, and how it was found

The Studio first served the model through oMLX v0.7.1rc1 (commit 7a5a3d7) on MLX 0.32.4, the server
Lab 020 used, with `mlx-community/gemma-4-31b-it-4bit` (revision 696d436c). Envelope matched to the
clustered arm: thinking off, sampling from generation config, 131,072 context. Every short gate
passed: tool round trip, decode 46.2 tok/s, salted cold prefill at 35.8K and 48.1K.

Run 1, the full pool, was stopped at 71 minutes with 0 of 22 done. The first two sessions had called
the same file read 603 and 628 times, each answered by the harness with "file unchanged since your
last Read," with no text between calls. Context was intact on every request (67K to 74K tokens). The
clustered arm's worst session had three such repeats.

Diagnostics, one task each (prettytable-pr332):

1. Prefix cache off (`--no-cache`), same weights: the loop reproduced within 11 calls.
2. A different file, `mlx-community/gemma-4-31b-it-nvfp4` (revision a0a935b), cache off: no loop.
   The turn after the first file read was a single 32,000-token reply of repeated nonsense, written
   in 25 minutes until it hit the output cap.
3. Direct probes on oMLX: planted-fact recall correct to 38.4K tokens at temperature 0 and 1.0, but
   a stray hyphen in place of a space in several short answers at every length ("a technical guide
   and-README", "features-continuous batching"). Both oMLX files showed it.
4. llama.cpp, the same task: 15 turns in 5.6 minutes, reading, editing and re-running tests, one
   repeated read. Not solved (it edited only a test file), the same verdict the clustered arm reached
   on that task in 84.7 minutes. In run 2 a few hours later, llama.cpp solved the same task in 12.9
   minutes: a borderline flip between two identical configurations.

Run 2 on llama.cpp followed. The oMLX defect was not isolated: server, chat-template handling and
the MLX kernels for this architecture all remain candidates.

## Apparatus changes

- The venue preflight read the served context only from `max_model_len`, which llama.cpp does not
  set; it reports `meta.n_ctx`. The preflight now falls back to that field. Behavior on vLLM and oMLX
  is unchanged.
- The bench runner counts the preflight stamp file in the results directory as a task record and
  prints "15 / 23". The 22 per-task records are the source: 15 of 22.

## What this hands the next labs

- The bandwidth threshold for practical dense agentic work sits between 273 GB/s and 1.2 TB/s. An
  M5 Max (614 GB/s) and a discrete GPU would place it.
- Run one real harness task on any new runtime before a pool, and read the session for repeated
  calls, not only the error log and the speed.

## Disclosure

Google Cloud is a client of this practice, and Gemma 4 31B is a Google model. Google did not
commission, fund, preview or have any say in this lab. Apple, NVIDIA and Anthropic are not clients.
Anthropic appears twice: Claude Code is the agent harness on every arm, and Claude Opus 5.5 drafted
the page essay from the record, which the author validated. No vendor paid for this answer.