Beyond CUDA: the lock was never the silicon

The workflow is portable. The same configuration file, the same trainer, the same serving container ran on AMD silicon with zero code changes, trained the same LoRA six times faster than the owned box, and the tuned model held quality parity under every instrument that survived scrutiny. CUDA is a default, not a requirement. What actually gates the non-CUDA path is two things this lab measured: who will sell you the silicon at all, and whether your acceptance gate is measuring what you think it measures.

Self-funded editorial with one comped input, disclosed: Hot Aisle’s CEO credited $100 to the account after it was opened, and the lab spent $29.75 of it. The credit funded the compute, not the answer. Hot Aisle did not commission, review, or see this lab before publication, had no say in what ran or shipped, and self-serve payment was available the whole time, so the credit never gated access. Correspondence with Hot Aisle is paraphrased where it appears. Disclosure: Google Cloud is a client of The CTO Advisor LLC, and it appears here twice: it failed the availability probe this lab reports, and it supplied one of the two judges (Gemini 2.5 Pro on Vertex AI, part of about $3 in self-funded API spend alongside AWS). Hot Aisle, AMD, Apple, NVIDIA, AWS, and Microsoft are not clients. The ruling is the author’s alone.

Lab 002 settled that the weights are the asset you keep. What it never tested was the ground it stood on. Every step of that ruling ran on NVIDIA silicon: the box trained, the box served, and even the cloud comparison was NVIDIA-backed. So the verdict carried a stowaway assumption. Does the workflow that produces and serves owned weights actually require CUDA, or is CUDA just where everyone happens to be standing? This lab reran the Lab 002 fine-tune on a rented AMD Instinct MI300X, brought the weights home to Apple silicon, and held everything else constant. Same base model, same training set, same frozen thirty questions, same recipe down to the learning rate. The silicon was the only variable.

The short answer is almost boring. The workflow is portable. The same configuration file, the same trainer, the same serving container ran on AMD hardware with zero code changes. CUDA turned out to be a default, not a requirement. The interesting findings sit on either side of the compute, and neither one is about the software moat everyone argues over. Below the compute: who will sell you the silicon at all. Above it: whether the acceptance gate you trust is measuring what you think it measures.

Who will actually sell you the card?

I asked four venues for AMD Instinct-class capacity in one afternoon. Google Cloud wouldn't provision anything with an accelerator attached. Amazon Web Services (AWS) rents NVIDIA parts fine, and this practice has rented L40S there, but its only AMD GPU is a 2020 graphics part. An L40S is not an answer to a 192GB high-bandwidth-memory question. Azure was the only hyperscaler that sells the MI300X, pricing it in 17 regions. It offered this account 2 of them, behind a family quota of zero. Three hyperscalers, three versions of no.

Then the neocloud. Hot Aisle published its shelf inventory in an API response. A GET request returned quantity on hand, a POST returned a running virtual machine, and the first inference token arrived 8 minutes after the provision call. Account-zero to a working MI300X shell took about 15 active minutes. The scope of that ladder matters: this is AMD Instinct-class silicon, not accelerators generally. The wall is specific to leaving CUDA. For silicon off the default path, access is a product surface, and the hyperscalers don't sell it. Don't infer capacity from a catalog. A catalog lies by omission.

Training became an errand

The Low-Rank Adaptation (LoRA) fine-tune that took 35 minutes on the Spark ran in 5.8 minutes on the rented card, for 29 cents, converging to the same loss over the same 111 steps. Identical recipe, identical data, six times faster. The 70B tier, the one that was relationship-gated out of existence in June, trained in 89 minutes for $8.89 across two cards. If you own the training set and the recipe, training compute is an errand, not a capital decision. The whole campaign spent $29.75 on GPU time, plus about $3 in judge APIs.

Here's where the obvious objection lands. Nobody got fired for buying NVIDIA, and the software moat is real. It is, at the frontier. Lab 002 hit a four-wall dependency cascade trying to fine-tune a brand-new architecture on CUDA, and this lab hit the same species of wall on ROCm: the stable container didn't know the newest model family, and the fix was a nightly image. The moat exists where the weights are newest. However, for the workload that pays the bills, a Llama-family LoRA on the standard stack, the moat has been abstracted away. PyTorch, the trainer, and vLLM installed clean on ROCm every time. And my two-Spark cluster needed a workaround the AMD boxes didn't: the graph-capture path hung under runtime LoRA on the NVIDIA stack while the ROCm container ran the identical configuration untouched. Day-N polish is workload-specific. Neither vendor owns it.

Serving came home the same way. The merged 8B served unquantized on the Mac at 26 tokens per second, outrunning the Spark that served the same weights in Lab 002. The dense 31B that crawls at 10 tokens per second across two clustered Sparks ran 5.6x faster on one rented card. At 64 concurrent streams that card delivered 1,974 tokens per second aggregate, about $0.42 per million output tokens on weights I own. Lab 002 found custom weights cost $27.50 per million on the managed floor. That floor was never physics. It was packaging.

The dual-card question answered itself along the way. A 70B fits whole on one 192GB card. The second card bought 1.6x single-stream speed for 2x the cost: a latency purchase, not an efficiency one. And the owned two-Spark cluster served the same tuned 70B at 3.2 tokens per second, which the rented single card beat 7.2x. Owned compute at this tier is a sovereignty floor, not a serving tier. The weights run at home on nobody's quota. When speed matters, you rent the hour.

The gate ate the headline

The quality comparison should have been the easy part. It wasn't, because the original judge script was lost and its judge model retired. I rebuilt the gate from the frozen contract and calibrated it against Lab 002's archived outputs. The rebuilt gate reads grounded more strictly: the archived 70% baseline scored 27% under it. Fine. Re-baseline and compare relatively. The AMD-trained 8B scored 33% against the Spark-trained original's 27%, parity within noise on a thirty-question set. The training venue cost nothing.

Then the strict gate said the 70B beats the 8B by 2.5x, and that number survived exactly until a human read the outputs. The wording difference isn't perceptible. Mechanical quote verification agreed with the human read: both models cite real quotes at parity, 99 and 100 percent, one fabricated citation out of 178. The judge pair wasn't detecting fabrication. It was expressing taste. If your acceptance gate is a model judge, you're buying the judge's taste, and the judge's taste can quietly pick your model tier for you.

So I ran four instruments over one set of outputs and got four verdicts. The lenient original gate said the 8B was saturated. The strict rebuilt gate said the 70B wins by 2.5x. Deterministic quote verification said parity. Human reading said parity. Honest bounds on all of it: one task in one domain, thirty questions, one neocloud tested, and a two-Spark number that is a floor of this stack version, not a ceiling. Directional, not a benchmark.

I went in to prove the training would run. It did, faster and cheaper than the box I own, and that stopped being the story by midnight. The story is that every gate I hit was a different kind than advertised. The software gate everyone fears didn't exist for this workload. The access gate everyone ignores decided which silicon I could even test. And the quality gate I built to referee it all turned out to need a referee. The question an Enterprise Architect actually asks, does the bigger model pay off, has no answer until you answer a prior one. Measured by what? Whoever defines the validator defines which tier wins, and nothing in the current tooling validates the validator. That question is still open. It's the one this lab keeps.

The numbers

Where each layer belongs

Assessments at the time of the lab

Method and disclosure

Self-funded editorial, no sponsor; the comped Hot Aisle credit is disclosed in the independence line. Every measurement ran under one frozen contract: same training set, same eval set, same retrieval index, same prompts as Lab 002. The only variable was silicon.

Training was Low-Rank Adaptation (LoRA) via the same trainer and recipe as Lab 002, on rented MI300X capacity. Serving was measured on the rented cards, an owned Mac (MLX, fp16), and the owned two-Spark cluster, with the same bench harness throughout. The judge gate was rebuilt from the frozen contract after the original was lost, calibrated against archived outputs, and then itself became a finding when mechanical verification and human reading overruled it.

The raw detail ships at the download link: the availability-ladder data, every bench table, the training configs, the complete friction log including the author’s own mistakes, and the cost ledger with the staging split. The training set, eval set, adapters, and judges stay with the practice.

The essay on this page was rendered from the lab’s frozen structured record by a model writing under this practice’s voice specification, with a deterministic gate checking every figure against the record before publication. The record is the canonical surface: where prose and record disagree, the record rules.