Renting the chip was the easy part

Scoped to the self-serve, on-demand path on a single GCP project, to the reachable v5e silicon (Trillium v6e capacity was dry on the measured date), and to the open-source vLLM-TPU serving stack. It is a ruling about the level of effort to serve a bring-your-own mid-size mixture-of-experts on the lane, not a claim that Google’s own stack cannot serve Gemma. The served model’s performance is referenced from Lab 006 and Lab 007, not re-measured here. This lab measured the friction, not the tokens per second.

Disclosure: Google Cloud is a client of this practice. Google did not commission, fund, preview, or have any input into this lab. The claim under test is Google’s own public positioning of the Tensor Processing Unit (TPU) as the price-performance home for Gemma-class inference, and the ruling was formed from roughly $25 of self-funded, on-demand probes. No vendor paid for this answer.

Google markets the Tensor Processing Unit (TPU) as the price-performance home for Gemma-class inference, and Lab 007 gave me a reason to believe the on-ramp. A chip in minutes, a quota bump in minutes, on the same project where the NVIDIA lane says no in two seconds. So this lab had a simple plan: put my mid-size Gemma 4 mixture-of-experts on the lane, fill a latency matrix, and price the pitch. The matrix never filled. Renting the chip turned out to be the easy part, and the level of effort between a rented chip and a served model became the finding. That gap is the thing the price-performance chart never mentions, and it's the thing a CTO actually plans around.

The model is the same one this whole arc carries: Gemma 4 26B-A4B, 25.2B parameters with 3.8B active, a mixture of experts I bring myself rather than pull from a garden. The target stack was the open-source vLLM-TPU path at bf16, because vLLM is what most teams already run, and the point of a bring-your-own model is that the serving stack comes with you. Performance wasn't re-bought here. Lab 006 already judged four-bit quality on this exact model and Lab 007 already priced the interactive frontier and the commodity API floor. This lab measured the friction, with timestamps, for roughly $25 of self-funded probes.

The shape that runs it is the shape you can't rent

Availability decided the silicon before any benchmark could. Trillium, the v6e generation, is Google's current flagship TPU and the shape a 26B model wants, because its 32GB chips hold at bf16 what a 16GB v5e chip can't. On the measured date, a v6e-4 create failed on insufficient capacity in all three catalogued zones. The quota was fine. The silicon wasn't there. And the shape matters, because Trillium is where this model demonstrably runs: an independent Google Developer Expert benchmark served the same 26B mixture-of-experts on a v6e-4 pod at about 457,000 tokens per second aggregate, first token under 1.2 seconds at low concurrency. That's an attributed community number, not mine and not first-party, and I cite it for one bounded claim only: the model has been served on TPU. So the lane's story checks out on hardware I could not rent, and what I could rent is v5e.

On v5e the bf16 model does not fit, and the model's own configuration explains why. Gemma 4 26B has two global key/value (KV) heads, which caps tensor parallelism (TP) at two; tp=4 and tp=8 fail to compile. At that degree the 51.6GB of bf16 weights shard across two chips at roughly 26GB each, against a 16GB budget per chip. The instinct says add chips. It doesn't work, because the open-source vLLM-TPU stack replicates the 128 experts instead of sharding them, and on a mixture-of-experts the expert weights are most of the model. Every chip carries nearly the whole thing no matter how many you rent. There is no tensor-parallel setting in this runtime that serves bring-your-own bf16 on this hardware. That's the whole ruling in one line: the silicon that serves the model was not rentable, and the rentable silicon does not fit it.

Two detours, both real, neither is rent-and-serve

Two paths do get you to a served model, and I want to be fair to both. The first is Google's own serving stack. MaxText and JetStream ship expert-parallel sharding for Gemma 4 that splits the 128 experts across chips, which is the route that fits bf16 on v5e. It's real and it's supported. But it isn't just another runtime. It's a move out of the Hugging Face and vLLM ecosystem into Google's: weights converted to its Orbax scanned-checkpoint format, its mesh config, its XLA compilation. A different software lineage, not a flag you flip. The second path is quantization. Four-bit weights fit a single chip, and Lab 006 already showed four-bit holds quality on this exact model: NVFP4 reached parity with bf16 and ran 25 percent faster, measured on the NVIDIA GB10, not on TPU. The catch is where you build the checkpoint. The single-chip v5e host has 47GB of RAM and the bf16 model is 50GB, so the host can't quantize its own model. You build off-box and ship the artifact.

Here's the objection I'd raise against myself: I gave up too early, because MaxText serves Gemma 4 on TPU fine, with day-zero support. Granted, fully. That concession is the point, not the rebuttal. This lab never claimed a TPU can't serve Gemma. It measured what it takes to serve a mixture-of-experts you brought yourself, on hardware you can actually rent self-serve, with the stack most teams already run. On that path the answer is blunt: it does not fit and does not serve, and reaching a served model means adopting the vendor's toolchain or running an off-box quantization pass. Both are fine engineering. Neither is rent-a-chip-and-serve-your-model, and the low-effort default on this lane is Google's stack, not yours.

The door isn't the problem

The quota system behaved well, and that sharpens the ruling instead of softening it. A serving quota increase from four to eight chips auto-approved in about 21 minutes, self-serve, no sales contact. Lab 007 already clocked a single chip reaching READY in about 17 minutes on a project with no TPU history. Of the three doors this arc has now measured, the TPU door is the best one. The friction isn't renting the chip. It's that a rentable chip and a servable model are two different things, and the gap between them is engineering I didn't have to do on a GPU. Lab 007 measured hardware portability. This lab measured runtime portability, and the runtime is where the model quietly changes hands: the same weights on the same silicon serve or fail depending on whether they run through vLLM or through Google's stack.

Bound it honestly, because the bounds are most of the value. Nothing served on TPU in this lab, so it has no TPU latency or throughput of its own; the v6e-4 figure is external and attributed, and the four-bit speed is Lab 006's, measured on NVIDIA silicon. MaxText and JetStream were never run. Whether that stack clears the interactive bar, and at what dollar-per-million, went unanswered here and stays unanswered in this corpus, because the capacity to bench it never materialized. The tensor-parallel cap and the expert replication are read from the model config and one vLLM-TPU image on one date; a future release that ships expert parallelism moves the ruling, and it's dated on purpose. The capacity finding is one project, one date, one self-serve path, not a market study.

One more confession, because verify-then-rent is cheaper than the reverse. Someone handed me a spec claiming Gemma 4 has four global key/value heads, which would cap tensor parallelism at four. The config said two. I almost rented a v5e-8 for an hour to prove a topology that can't compile. Verify the config before you rent the chip. I keep relearning that one. And the image I can't shake is the quota bump approving in 21 minutes next to a model that still wouldn't fit. The door isn't the problem. The room behind it is. Whether the Trillium drought is a bad week or a structural fact is the question this lab leaves standing, and it's answerable the same way Lab 007 answered the CPU lane: a capacity census, re-run until the anecdote becomes a trend line.

The numbers

How the lanes compare

Where each layer belongs

Assessments at the time of the lab

Method and disclosure

Editorial and self-funded: roughly $25 total, on-demand rates, no vendor involvement. Disclosure: Google Cloud is a client of this practice; this lab was not commissioned, funded, or previewed by Google or any vendor, and the ruling was formed from the probes alone. The claim under test is Google’s public positioning of TPU as the price-performance home for Gemma-class inference. An editorial lab discloses the relationship and rules anyway.

What was measured here: TPU provisioning and quota behavior with timestamps, the model’s tensor-parallel ceiling from its own configuration, the bf16 memory fit on v5e, and the vLLM-TPU expert-replication behavior on a nightly image, all on a single GCP project on the dated run. What was referenced, not re-measured: the v6e-4 throughput for the same mixture-of-experts, four-bit quality and throughput from Lab 006, and the interactive frontier and commodity API floor from Lab 007. No TPU serving latency was measured, because the level of effort to reach a served model was the finding. The custom adapter, its training data, and the corpus taxonomy are proprietary and appear nowhere.

One external source is cited, with attribution: an independent Google Developer Expert benchmark of the same 26B mixture-of-experts on v6e-4. It is used to establish that the model has been served on TPU, not as a first-party or peer-reviewed result. This is an editorial lab, not a sponsored or academic one, so an attributed community benchmark clears the bar for that single, bounded claim. A dense-model result would not have, because the difficulty here was the mixture-of-experts specifically.

The essay on this page was rendered from the lab’s frozen structured record by a model writing under this practice’s voice specification, with a deterministic gate checking every figure against the record before publication. The record is the canonical surface: where prose and record disagree, the record rules.