{ "slug": "mac-memory-tiers", "lab_number": 21, "title": "Memory decides which model can do the work.", "type": "editorial", "status": "published", "sponsor": null, "author": "Keith Townsend", "date": "October 18, 2026", "date_iso": "2026-10-18", "layers": [ "layer0", "layer2c" ], "axes": [ { "instrument": "4plus1", "layers": [ "layer0", "layer2c" ], "has_dapm_table": true } ], "supersedes": [], "superseded_by": null, "corrects": [], "corrected_by": [], "vendors": [ { "key": "apple", "role": "hw" }, { "key": "google", "role": "model" }, { "key": "alibaba", "role": "model" }, { "key": "nvidia", "role": "hw" }, { "key": "anthropic", "role": "harness" } ], "themes": [ "serving-economics", "small-model-viability", "operating-cost-of-ownership" ], "finding": "On Apple silicon, unified memory decides which model can run an agentic coding harness, and the model decides the score. Gemma 4 26B fit both a 36 GB M5 Max and a 256 GB M5 Ultra, scored 1 of 22 on the Max because the memory guard ended 20 of 22 sessions, and hit no memory wall on the Studio. Qwen3.8-Flash-Next, the model that actually solved the tasks (20 of 22 on the Studio), needs about 105 GB loaded. Inside an emulated 128 GB M5 Max it was refused at the stock limit and scored 20 of 22 only with the wired limit raised to 120 GB, leaving about 8 GB for macOS and every app. The Studio ran it while its owner used it as a normal desktop, an observation rather than an instrumented measurement.", "video": null, "lab_detail": "https://labs.layer2c.com/downloads/mac-memory-tiers-lab-detail.md", "question": "Which Mac does an architect buy for a local coding agent: a 36 GB laptop, a 128 GB laptop, or a 256 GB Studio? The usual answer is a decode-speed chart. This lab ran the same 22 verified repair tasks, under the same agent harness, on a 36 GB M5 Max and a 256 GB M5 Ultra, with one model that fits both and one model that does the work. Then it asked whether a 128 GB M5 Max, the largest laptop configuration, could carry the model that works.", "load": "The certified 22-task pool of Python repair tasks, each scored by its own held-out tests in a sealed container, driven by Claude Code at two concurrent sessions and a 7,200-second ceiling per task. Both Macs served through oMLX v0.7.1rc1 on MLX 0.32.4, pinned to the same commit and package freeze. Models: Gemma 4 26B-A4B at 4 bits (14 GB) and Qwen3.8-Flash-Next at 4 bits (111.5 GB). Before each pool, an envelope probe measured memory at one to eight sessions and up to 250,000 tokens of context. The 128 GB laptop was emulated on the Studio by capping the server process at that machine’s limit.", "verdict": { "scope": "Two Macs, two models, one harness, one replicate per cell, over two days. The memory figures are measured on the Studio and the 36 GB Max. The 128 GB M5 Max is emulated, not measured: the cap models the server process only, the Studio’s own apps sat outside it, and the Studio has about twice that laptop’s GPU cores, so its wall clock flatters a real one. The two-solve swaps between cells sit inside temperature-1.0 variance on identical weights.", "independence": "Self-funded editorial. No part of this lab was sponsored, and no vendor commissioned it, funded it, previewed it or saw it before publication. Both Macs were bought by the author at retail. Disclosure: Google Cloud is a client of The CTO Advisor LLC, and Google’s Gemma 4 is one of the two models here. Apple, Alibaba and NVIDIA are not clients. Anthropic’s Claude Code is the harness in every arm, and an Anthropic model drafted this record.", "calls": [ { "kind": "dont", "text": "buy a 36 GB Mac to run a coding agent locally. The 14 GB model that fit didn’t fit beside a normal day’s apps at the guard tier the Studio used, and with everything closed it still couldn’t carry two sessions. Five sessions died before their first turn because the other session held the memory, and the memory guard ended 20 of 22. The machine scored 1 of 22 on weights that scored 14 of 22 on a DGX Spark in Lab 013 and 15 of 22 in Lab 018. That is memory, not capability." }, { "kind": "do", "text": "pick the model first and size the memory to it. The model that did the work, Qwen3.8-Flash-Next at 4 bits, scored 20 of 22 and needs about 105 GB loaded plus about 13 GB per full-context session. The model that fit everywhere, Gemma 4 26B, never got past 1 of 22 on the Max and stalled in loops and runaway generations on the Studio. On this workload the score belongs to the model, and memory decides only whether the model can load." }, { "kind": "dont", "text": "read a 128 GB laptop as a smaller Studio, or a 96 GB Studio as a faster 128 GB laptop. At its stock limit it can’t take a 55,000-token prompt with Flash-Next loaded. With the wired limit raised to 120 GB it ran one session at full score, with about 8 GB left for macOS and every app, which is less than this author’s ordinary apps used on the smaller Mac. That is a dedicated inference machine, not a laptop you also work on. The 96 GB M5 Ultra costs about the same and can’t load that model at all. Its extra GPU cores only pay if a model under about 60 GB does the work and the Ultra clears the memory-speed floor for it." }, { "kind": "do", "text": "budget disk for the prefix cache, and keep the peak apart from the total. The workload runs as decode-heavy only because 98 to 99 percent of each session’s re-sent context comes back from cache. oMLX sized that cache to about half the free disk. It peaked at about 1.2 TB on the Studio and wrote at least 2.0 TB over the lab, against 367 GB on the Max. Peak is a capacity question and the total is an SSD-wear question, and neither scales with RAM." } ] }, "objection": { "q": "This is a decode-speed contest with extra steps. Faster memory, faster tokens, more solved tasks.", "a": [ "Faster memory did deliver faster tokens: the Studio decoded Gemma at 165.6 tokens per second per stream against the Max’s 111.6, and prefilled 128,000 tokens in 38 seconds against 89. It didn’t deliver more solves. Gemma scored 0 of the 7 tasks the Studio ran before the run was stopped as redundant, including one that Lab 013’s Spark arm had solved, which got a full attempt of 50 turns. Speed doesn’t change what a model can reason through: a faster machine samples from the same weights the same way. The corpus has no example of speed raising a score except by finishing work the clock would otherwise have cut off.", "What memory changed was whether a session survived and which model could load. On the Max, 20 of 22 sessions ended on a memory refusal, including five that never took a turn. On the Studio none did. And the model that scored 20 of 22 doesn’t load on the Max at any size, and fits a 128 GB laptop only with the wired limit raised. Speed can only give back tasks a ceiling truncated while they were on track. Memory decides whether the session gets to run at all." ] }, "numbers": [ { "label": "Gemma 4 26B pool, 36 GB Max", "value": "1/22", "note": "20 of 22 sessions ended by the oMLX memory guard, 5 before the first turn; 13 misses on the 14 tasks this model solved on a DGX Spark in Lab 013 (14/22); Lab 018 published 15/22" }, { "label": "Gemma 4 26B, 36 GB Max, safe tier", "value": "refused at load", "note": "8.2 GB ceiling with apps open, 14.5 GB with apps closed, against 15.0 GB of weights; ran at the balanced tier (8% reserve), the proportional match to the Studio’s safe (16 of 256 GB)" }, { "label": "Gemma 4 26B, eight sessions at 250K, Studio", "value": "94.8 GB peak", "note": "no memory wall; prefill serializes, so the eighth session waited 1,066 s for a first token" }, { "label": "Decode, Gemma, one stream, per-stream rate", "value": "111.6 vs 165.6 tok/s", "note": "Max vs Studio; 8 streams aggregate 271.6 vs 539.2. Diagnostics, not the result" }, { "label": "Qwen3.8-Flash-Next pool, Studio, two sessions", "value": "20/22 · 1 h 25 m", "note": "no timeouts, no memory events; 7 of the 8 tasks Gemma 4 26B missed in Lab 013" }, { "label": "Flash-Next memory, Studio", "value": "105 GB loaded · ~121 GB at 250K", "note": "phys_footprint 104.95 GB loaded; about 13 GB per full-context session; four sessions at 250K peaked at 166.3 GB wired" }, { "label": "Emulated 128 GB Max, stock 112 GB", "value": "refused at preflight", "note": "a 55K prefill needed 106.48 GB against a 106.40 GB prefill limit" }, { "label": "Emulated 128 GB Max, wired limit 120 GB, one session", "value": "20/22 · 1 h 43 m", "note": "peak 115.5 GB, largest prompt 142,682 tokens; leaves about 8 GB for macOS and apps" }, { "label": "96 GB M5 Ultra vs 128 GB M5 Max, Flash-Next 4-bit", "value": "no vs one session", "note": "capacity arithmetic from measured footprints, at about the same price; the 128 GB session needs the wired limit raised. Full matrix 36 GB to 512 GB in the raw detail" }, { "label": "Prompt tokens served from prefix cache", "value": "98–99%", "note": "both Macs; decode is 54–89% of compute time with runaway generations excluded" }, { "label": "oMLX SSD prefix cache, Studio", "value": "~1.2 TB peak · ≥2.0 TB written", "note": "auto sizing took about half the free disk; Max 367 GB. Peak is capacity, the total is SSD wear" } ], "opened": [ { "q": "Where is the memory-speed floor for a dense model on a Mac?", "a": "A model’s score carries across platforms once the platform clears a floor: enough RAM to hold the model and its sessions, enough memory speed to finish turns inside the harness ceiling, and a serving stack that serves the model correctly. Lab 018 showed the last one matters: a tool-call parser bug made Gemma 4 26B look incapable, and with the correct parser it scored 15 of 22. A dense model that fits both the 96 GB Ultra and the 128 GB Max reads every weight for every token, so speed is where it can fall short. Lab 013 recorded Gemma 4 31B dense at 11 of 22 on an L40S, and Lab 017 measured the same model at 10 tokens per second on two clustered Sparks. Where Apple silicon sits against that floor for a dense model wasn’t measured here." }, { "q": "What does a real 128 GB M5 Max do at 8 GB of headroom?", "a": "The emulation capped the server process and nothing else. The Studio’s own apps and macOS sat outside the cap, and the Studio has about twice the laptop’s GPU cores. Whether a real 128 GB laptop stays responsive, swaps, or wedges with 8 GB left for everything else, and how long the pool takes at half the cores, needs the laptop." }, { "q": "How much does PLE offload buy on a 128 GB machine?", "a": "Flash-Next carries per-layer embedding tables that oMLX can move to SSD. That could shrink the resident weights enough to fit the stock 112 GB envelope, at a cost in decode speed nobody has measured. It would be a different instrument, an SSD-bound one, and this lab didn’t run it." } ], "not_proved": [ "It didn’t measure a 128 GB M5 Max. The 128 GB cells are an emulation on the Studio: memory capped, compute not, apps outside the cap. Read them as the memory answer only.", "It didn’t compare Flash-Next across venues. There is no GPU record of Flash-Next on this pool, so its 20 of 22 has no cross-venue comparison here.", "It didn’t separate replicate noise. Every cell is one run at temperature 1.0. Flash-Next swapped which two tasks it missed between its two cells.", "It didn’t finish Gemma on the Studio. That run stopped at 7 of 22 by the author’s call once the comparison was clear, and a first Studio run is stranded at 5 of 22 on a Spark that stopped answering. Neither changes a finding.", "It didn’t measure memory bandwidth. Decode differences are reported as measured rates, not attributed to a specification-sheet figure." ], "executive_summary": [ "Two Macs with identical file systems and applications ran the same agentic coding pool. The M5 Max has 36 GB of unified memory and 32 GPU cores. The M5 Ultra Mac Studio has 256 GB and 80. Both served through the same pinned oMLX build, and the harness ran on a DGX Spark, so the only thing that moved was the Mac.", "Gemma 4 26B at 4 bits is 14 GB, small enough to load on both. On the Max it didn’t load at first. The oMLX memory guard at its safe tier refused it with a normal day’s apps open, and refused it again with everything closed but the operator’s own tools, short by half a gigabyte. At the next tier down, the envelope probe served one session at 128,000 tokens, refused one session at 250,000, and aborted two sessions at 128,000 mid-prefill. The pool scored 1 of 22. The memory guard ended 20 sessions, five of them before the first turn, and 13 of the misses are tasks this model solved on a DGX Spark in Lab 013.", "The Studio served the same weights at the same safe tier with its apps left open, against a 210.8 GB ceiling. Eight concurrent sessions at 250,000 tokens peaked at 94.8 GB. There was no memory wall. Its limit was time: oMLX prefills one request after another, so the eighth session waited 1,066 seconds for its first token. Gemma’s own limits showed through on both Macs: tool-call loops (one repeated a single command 266 times on the Max) and runaway generations to the 32,000-token output cap. Those are Gemma’s, and they appear in the GPU records too.", "Qwen3.8-Flash-Next is a sparse mixture of experts with 512 experts, 10 active per token. At 4 bits it’s 111.5 GB on disk and 104.95 GB loaded. It doesn’t load on the 36 GB Max at any published size. On the Studio it scored 20 of 22 in 1 hour 25 minutes, with no timeouts and no memory events. Of the 8 tasks Gemma 4 26B missed in Lab 013’s 14 of 22, it solved 7, and the 120 GB cell below solved all 8.", "The 128 GB laptop question was emulated on the Studio by capping the server process. At 112 GB, the stock safe ceiling for that machine, Flash-Next loaded and then couldn’t take a 55,000-token prompt, so the bench gate refused the arm. At 120 GB, the ceiling with the wired limit raised by the owner, it scored 20 of 22 at one session in 1 hour 43 minutes, peaking at 115.5 GB with the largest prompt at 142,682 tokens. That leaves about 8 GB for macOS and every app." ], "dapm": [ { "instrument": "4plus1", "reading": "portability", "layer": "layer0", "placement": "Retained", "cells": { "p": "Retained" }, "note": "Retained, because the workload isn't tied to Apple's hardware. The same Gemma weights ran under vLLM on DGX Sparks in prior labs and under oMLX on both Macs here, behind the same OpenAI-compatible interface, with no harness change. MLX itself runs only on Apple silicon, so the serving runtime is per-venue. The memory ceiling that decides which model loads is a property of the box you buy, and that is the purchase question this lab answers, not a portability one." }, { "instrument": "4plus1", "reading": "portability", "layer": "layer2c", "placement": "Retained", "cells": { "p": "Retained" }, "note": "Retained, because both models are open weights served from files the enterprise holds, at a pinned revision, on hardware it chose. The same checkpoint was copied between Macs and SHA-verified. Nothing a vendor runs decides how the agent reasons, and the deterministic test gate decides when a task is done." } ], "seam_map": [], "writeup": [ "The lab asked a buying question and refused to answer it with tokens per second. Two Macs, one harness, one task pool, two models. The 36 GB M5 Max is a well-specified developer laptop. The 256 GB M5 Ultra Mac Studio is the author’s. That configuration lists at $12,299. Both ran the same pinned oMLX build on the same MLX version, and the harness ran on a DGX Spark so that model-written code never executed on a personal machine. The unit was verified solves: a task counts when its own held-out tests pass in a sealed container.", "The model that fits both boxes was chosen first. Gemma 4 26B at 4 bits is 14 GB, and it carries published records on this exact pool and harness: 14 of 22 as NVFP4 on one Spark in Lab 013, and 15 of 22 in Lab 018 once its tool-call parser was fixed. That gave a model whose capability was already on file, so anything the Max lost could be checked against it.", "On the Max, the first result arrived before a probe ran. oMLX runs a memory guard that refuses work it predicts won’t fit. At the safe tier, which holds back 20 percent of RAM, it refused to load the model with a normal day’s applications open, and again with everything closed but the operator’s own tools: the desktop Claude app, Claude Code and Spotlight indexing, about 6 GB nobody is going to quit. The Studio’s safe tier holds back 16 of 256 GB. The Max ran at the balanced tier, 8 percent, as the proportional match. On a workstation-class box the binding resource wasn’t the model or the cache. It was everything else the person at the keyboard was running.", "The envelope probe then found the wall. One session at 128,000 tokens served. One session at 250,000 was refused at preflight, needing about 30 GB against a 22.8 GB prefill limit. Two sessions at 128,000 were admitted and then aborted mid-prefill when the process crossed its hard watermark, and the guard unloaded the model. At the harness’s two concurrent sessions the pool collapsed in a specific way. Twenty of 22 sessions ended on a guard refusal. Five never took a turn, because their opening request of 28,000 to 61,000 tokens arrived while the neighbor held the memory. The harness recorded each guard-ended miss as a model that made no edit. A memory abort on this venue scores as a model that failed, unless someone reads the session logs.", "The Studio served the same weights, at the safe tier, with its apps left open, and the author used it as an ordinary desktop the whole time. That is an observation, not an instrumented responsiveness test, and it covers this lab’s pools, not heavier footprints such as Lab 020’s DeepSeek pool. Eight concurrent sessions at full context peaked under 95 GB. The Studio’s limit was time, not memory. oMLX prefills one request after another, so concurrency on long prompts buys a queue. Gemma’s run there was stopped at 7 of 22 as redundant, with no memory events. Its misses were Gemma’s own: tool-call loops and runaway generations, which the GPU records show as well.", "So the question moved to the model. Qwen3.8-Flash-Next is a sparse mixture of experts, 10 of 512 experts active per token, 104.95 GB loaded at 4 bits. On the Studio it scored 20 of 22 in under an hour and a half. It solved 7 of the 8 tasks Gemma 4 26B missed in Lab 013, in 8 to 25 minutes each. It doesn’t load on the 36 GB Max at any published size. To answer the 128 GB laptop question, the Studio stood in for one with the server process capped. At 112 GB, the stock limit, a 55,000-token prompt was refused before the first task. At 120 GB, with the wired limit raised, the pool scored 20 of 22 at one session. The process peaked at 115.5 GB and touched the guard’s soft mark 316 times without a refusal. That configuration leaves about 8 GB for macOS and every app.", "The matrix in the raw detail carries these footprints across the memory options, from 36 GB to 512 GB. It reproduces every measured point, and the close call is the 96 GB M5 Ultra against the 128 GB M5 Max, at about the same price. They split on the model, not on speed. The 128 GB Max is the only one of the two that holds Flash-Next at 4 bits, and only as a dedicated machine. The 96 GB Ultra can’t load it, and holds the untested 2-bit build only with its wired limit raised. For a model that fits both, the Ultra brings 64 GPU cores against 40 and the Max brings more room for sessions beside the desktop. Session counts are capacity, not throughput, because prefill on these machines runs one request at a time.", "Two measurements shaped how to read all of it. First, the workload is decode-heavy, but only because of the prefix cache. An agentic session re-sends its whole context every turn, and 98 to 99 percent of those tokens came back from cache: 63 million reused against under a million newly prefilled on the Max. Without the cache, re-prefill would have cost about 12 hours on the Max and 7 on the Studio. That is why cold-prefill benchmarks miss this workload, and it is also where memory comes back in, because the cache lives in memory and the Max’s refused requests were all cold prefills. Second, oMLX’s default SSD cache sizing took about half the free disk. It peaked at about 1.2 TB on the Studio and wrote at least 2.0 TB over the lab, with nothing in the log to warn about it." ], "abstraction": [], "method": [ "Self-funded editorial. The instrument is loopcontrolbench, open under MIT: the 22-task certified pool, the sealed test gate, and the Claude Code harness driver, run at two concurrent sessions with a 7,200-second ceiling except where a cell says otherwise. The harness ran on DGX Sparks, never on the Macs, because the Macs sit on a personal network and the harness executes model-written code.", "Both Macs served through oMLX v0.7.1rc1 at commit 7a5a3d75 on MLX 0.32.4, installed from one package freeze. Custom kernels were built on the Studio and absent on the Max, and none of them target either model here. Each instance ran with its own state directory and a single model, so the bench’s envelope gate read the served context unambiguously. Sampling came from each checkpoint’s generation_config. Requests reached oMLX through its Responses API on both Macs.", "Declared confounds, one line each. macOS 26.6.2 on the Max against 27.0.1 on the Studio. The Max ran at the balanced guard tier and the Studio at safe, matched by proportion of RAM reserved. Gemma’s prior records ran NVFP4 under vLLM on a Spark in Lab 013, plus Lab 018’s corrected-parser run, against MLX 4-bit here. The Studio’s Gemma and Flash-Next cells ran their harness on a second Spark of the same class after the first stopped answering. The two copies of the harness differ only in two bookkeeping fields.", "The 128 GB cells capped the oMLX process with its custom memory guard: 112 GB is 128 minus oMLX’s 16 GB safe reserve, and 120 GB models the owner raising the wired limit. Apple’s default GPU working-set cap on a 128 GB machine wasn’t measured. On these two Macs it read 78 and 87 percent of RAM.", "The same Studio served DeepSeek V4 Flash through this harness at 22 of 22 (`sf_dsv4_studio_r1`), which is Lab 020’s measurement, not this one’s. Lab 020 compares the Studio against two DGX Sparks. This lab compares it against a laptop.", "The raw detail ships at the download link: every arm’s configuration, the envelope probes, the per-task tallies, the guard events, and the cache accounting.", "The essay on this page was drafted by Claude Opus 5.5, an Anthropic model, from this record under the author’s voice specification, passed the deterministic post gate, was checked by a second model from a different vendor, and was approved by the author before publication. Anthropic’s Claude Code is the harness in every arm here, so the drafting model’s vendor is also part of the apparatus." ], "assessments": [], "publisher": "The Advisor Bench LLC", "program": "Layer2C Labs", "url": "https://labs.layer2c.com/labs/mac-memory-tiers", "markdown_url": "https://labs.layer2c.com/labs/mac-memory-tiers.md", "json_url": "https://labs.layer2c.com/labs/mac-memory-tiers.json" }