# Renting the chip was the easy part

> Lab 008 · Editorial lab · Status: published  
> Published by The CTO Advisor LLC · Layer2C Labs

**Question:** Google markets the Tensor Processing Unit (TPU) as the price-performance home for Gemma-class inference, and Lab 007 showed the door opens fast: a chip in minutes, a quota bump in minutes, where the NVIDIA lane says no in seconds. This lab set out to serve a mid-size Gemma 4 mixture-of-experts on the lane and fill a latency matrix. It never filled the matrix, because renting the chip turned out to be the easy part. On the silicon you can actually rent self-serve, a bring-your-own model does not fit, and getting it to serve means adopting Google’s stack or quantizing off-box. The performance was already trusted work in earlier labs. The friction was the finding.

**Load:** Gemma 4 26B-A4B (25.2B parameters, 3.8B active, mixture of experts), targeted for serving on GCP TPU. Attempted on v5e, because Trillium (v6e) capacity was dry in every catalogued zone on the measured date, with the open-source vLLM-TPU stack at bf16. The provisioning and quota door was measured with timestamps, and the model’s tensor-parallel ceiling was read from its own configuration. Performance is referenced, not re-measured: four-bit quality and throughput from Lab 006, the interactive frontier and the commodity API floor from Lab 007. The two workload shapes and concurrency sweep were the intended matrix; they are carried by reference, because the model did not serve on the reachable path.

## Executive Summary

The pitch under test is Google’s positioning of the Tensor Processing Unit (TPU) as the price-performance home for Gemma-class inference. Lab 007 had already shown the door opens: a single chip reached READY in about 17 minutes on a project with no TPU history, while the NVIDIA lane denied its requests in two seconds. This lab asked the next question, whether a mid-size mixture-of-experts you bring yourself actually serves on the lane, and it stopped buying answers the moment the effort became the answer.

The reachable self-serve silicon is v5e, because Trillium (v6e) capacity was dry in every catalogued zone on the measured date. On v5e the bf16 model does not fit. Gemma 4 26B has two global key/value heads, which caps tensor parallelism at two, and at that degree the weights land on two chips at roughly 26GB each against a 16GB budget. Adding chips does not help, because the open-source vLLM-TPU stack replicates the 128 experts rather than sharding them. There is no tensor-parallel degree that serves bring-your-own bf16 on this 16GB hardware. On the 32GB Trillium chips it does fit, and an independent benchmark clocks the same mixture-of-experts there at about 457,000 tokens per second. Trillium was the shape that came back capacity-dry.

Two paths get you to a served model, and both are detours. Run Google’s MaxText and JetStream, which ship expert-parallel sharding and are the supported route that fits, at the cost of adopting Google’s checkpoint format, mesh config, and compiler. Or quantize to four bits off-box and serve that on vLLM, which Lab 006 already showed is quality-neutral on this model. The single-chip host can’t even do the quantizing itself. It has 47GB of RAM against a 50GB model.

So the ruling is about level of effort, not tokens per second, and the performance is referenced rather than re-bought: Lab 006 for four-bit quality and speed, Lab 007 for the interactive frontier and the $2.50 commodity floor. The chip is self-serve and the quota negotiates in minutes. Serving your own model is where the lane charges you, and it charges in engineering.

## DAPM Table — Authority Verdict

| Layer | Placement |
| --- | --- |
| layer0 | Delegated |
| layer2c | Retained |

## Detailed Writeup

The bench was designed to serve the model and fill a latency matrix. It became a record of why the matrix does not fill on the reachable path, which is the more useful result. Every wall is documented with a timestamp, and together they measure the level of effort the price-performance chart leaves out.

Start with availability, because it decides the shape. Trillium is Google’s current flagship TPU generation, the shape a 26B model wants, and on the measured date a v6e-4 create failed on insufficient capacity in all three zones that list it. The quota was fine. This is the same capacity-not-policy pattern Lab 007 found at single-chip scale, now at the multi-chip shape a 26B model needs. The lane that grants a chip in minutes did not grant this shape at all. And the shape matters, because Trillium is where this model actually runs. An independent Google Developer Expert benchmark served the same 26B mixture-of-experts on a v6e-4 pod at about 457,000 tokens per second aggregate, with first token under a second and a half at low concurrency. The 32GB Trillium chips hold at bf16 what the 16GB v5e chips cannot. So the silicon that serves the model is precisely the silicon that was not rentable, and the reachable silicon does not fit it. That is the whole ruling in one line.

That forces v5e, and v5e forces the memory question. The model’s own configuration answers it. Two global key/value heads cap tensor parallelism at two, so bf16 shards to two chips at about 26GB each and a 16GB chip runs out of memory on load. The instinct is to add chips. It doesn’t work, because the serving stack replicates the experts. On a mixture-of-experts the expert weights are most of the model, and replicated experts mean every chip carries nearly the whole thing regardless of the chip count. Expert parallelism is the fix, and expert parallelism is Google’s MaxText, not the open-source vLLM path. That is the boundary this lab actually found, and it is worth naming plainly: the TPU itself is not the portability wall, the serving runtime is. The same weights on the same silicon serve or fail depending on whether they run through vLLM or through Google’s stack. Lab 007 measured hardware portability. This one measured runtime portability, and the runtime is where the model quietly changes hands.

The quota door, by contrast, behaved well, which sharpens the ruling rather than softening it. A serving quota increase from four to eight chips auto-approved in about 21 minutes, self-serve, no sales contact, the same negotiable behavior Lab 007 saw on the CPU lane and the opposite of the NVIDIA policy wall. So the friction is not the door. A CTO can rent a TPU faster than a GPU and negotiate its quota in minutes. What the CTO cannot do, on this hardware with this stack, is drop a bring-your-own mid-size mixture-of-experts onto it and serve. That takes the vendor’s toolchain or an off-box quantization pass, and either one is the level of effort this lab set out to name.

## Assessments at the Time of the Lab

| Vendor | Layer | Grade | As assessed |
| --- | --- | --- | --- |
| Google Cloud AI Infrastructure | Layer 0 · Compute | TPU + GPU Full Stack | July 12, 2026 |

## Method and Disclosure

Editorial and self-funded: roughly $25 total, on-demand rates, no vendor involvement. Disclosure: Google Cloud is a client of this practice; this lab was not commissioned, funded, or previewed by Google or any vendor, and the ruling was formed from the probes alone. The claim under test is Google’s public positioning of TPU as the price-performance home for Gemma-class inference. An editorial lab discloses the relationship and rules anyway.

What was measured here: TPU provisioning and quota behavior with timestamps, the model’s tensor-parallel ceiling from its own configuration, the bf16 memory fit on v5e, and the vLLM-TPU expert-replication behavior on a nightly image, all on a single GCP project on the dated run. What was referenced, not re-measured: the v6e-4 throughput for the same mixture-of-experts, four-bit quality and throughput from Lab 006, and the interactive frontier and commodity API floor from Lab 007. No TPU serving latency was measured, because the level of effort to reach a served model was the finding. The custom adapter, its training data, and the corpus taxonomy are proprietary and appear nowhere.

One external source is cited, with attribution: an independent Google Developer Expert benchmark of the same 26B mixture-of-experts on v6e-4. It is used to establish that the model has been served on TPU, not as a first-party or peer-reviewed result. This is an editorial lab, not a sponsored or academic one, so an attributed community benchmark clears the bar for that single, bounded claim. A dense-model result would not have, because the difficulty here was the mixture-of-experts specifically.

---
*Layer2C Labs · The CTO Advisor LLC · labs.layer2c.com*