Own the weights, or the platform owns you

The economics are arithmetic once you know the floor, and they sit below as evidence. These are the rulings: where authority goes when you customize, and what each managed layer takes in exchange for convenience. One caveat, not a hedge: the box that wins here is a contained, air-cooled, plug-in system. This is not a case for on-prem at scale, where cooling, power, and facilities re-enter the math and this lab did not go.

Self-funded. No vendor paid for this answer.

Lab one ended on a verdict I could have left alone: the NVIDIA DGX Spark loses to the cloud on inference. This lab asked whether that verdict survives the moment a workload needs a custom model you own. It doesn't. The lab-one ruling was measured against commodity per-token pricing, and that pricing is a property of base models, not of the cloud. Fine-tune the model and the token price is gone. What remains are floors: Provisioned Throughput billed by the hour, continuously, or Custom Model Import billed by the active window. Against a floor, a flat monthly box wins on cost above a low utilization line. The Spark never lost to the cloud. It lost to commodity tokens, and custom weights are exactly where commodity tokens do not exist.

I should say up front that the lab refused to be the lab I planned. I set out to scale the box up to a GB300-class cluster and move the crossover line outward. The access walls and the memory math killed that on the first day, and the question that was actually unanswered was the one about custom models. So I let the lab pivot rather than force the plan. That is what a lab is for, and the richest results came from the friction, not the compute.

Fine-tuning had to earn its place first

The kill-criterion was built in: if base model plus retrieval clears the bar, do not fine-tune. It did not clear. Base plus retrieval scored 23% accepted output on the Spark and 47% on Bedrock against the frozen question set, and the failures were grounding, not formatting, so few-shot prompting and constrained decoding would not have closed them. A free local Low-Rank Adaptation (LoRA) fine-tune, thirty-five minutes on the box, took accepted output to 70%, three times the local baseline and better than the base model served on Bedrock, with training loss falling 0.34 to 0.22. The custom model was worth building. And those weights are mine to serve anywhere.

Capability, though, was not the differentiator, and that surprised me. Quality saturates at the task ceiling; a 4B model ties a 14B on a bounded task. Speed is the axis instead. A 26B mixture-of-experts (MoE) model with roughly 4B active parameters runs 1.6 times faster than a dense 8B on the box, 22.4 tokens a second against 14, because single-stream decode on bandwidth-bound hardware is set by active parameters, not total. That advantage is independent of fine-tuning. So the self-host optimization is the fastest model that clears the task ceiling, not the biggest.

Data goes in, no model comes out

The obvious objection is that the cloud is cheaper. It is, for base models. Commodity pricing, about forty cents per million output tokens, exists only for a model you did not change. The cloud will not sell you cheap tokens on weights you own. It rents you a floor instead, and the floors are the whole story. Custom Model Import bills active windows and scales to zero, about $27.50 per million tokens for this behavior, and it is fast: 95 tokens a second on cloud GPUs, 6.8x the Spark. Native fine-tune serving is Provisioned Throughput, billed continuously with no scale-to-zero. Against those floors, the owned box at a flat $160 to $275 a month wins custom serving above roughly 12 to 20% utilization, lower for the faster MoE. Below that line, the cloud's scale-to-zero wins. It is a utilization line, not an hours line, and once you know the floor the math is arithmetic.

The authority finding is sharper than the arithmetic. Bedrock native fine-tune is a one-way door with three compounding traps: serving is quota-walled, where a no-commitment endpoint returns zero model units and a support case; it bills continuously with no scale-to-zero; and the job output contains metrics only, with no way to download the weights. Data goes in and no model comes out. There is no train on the managed platform and serve on your own import, because the paths do not bridge. The real axis is not price. It is ownership of the weights, and the managed path keeps them. Pay to shape the model and you have still ceded it.

Lab one found that managed retrieval hides the chunking decision. This lab found that managed fine-tune keeps the weights, managed hosting dictates the supported shapes, and the open stack lags the newest architecture. That last one bit directly: fine-tuning the latest model hit a four-wall dependency cascade, where the released transformers library could not load it, one trainer broke on the next transformers major version, and another needed a torchao past what the stable box could hold. The model is servable on the box and not fine-tunable on the stable stack without bleeding-edge tooling. The blocker is not the hardware. Every managed layer trades one specific control for its convenience, one layer deeper each time, and the fallback for anything outside the supported shape is always the primitive: your own box, or raw compute you build on yourself.

The wall moved from throughput to availability

The scale-up I abandoned taught its own lesson. Renting a GB300-class node was not a form, it was a relationship. GPU quota sat at zero across every project. A self-serve request approved in sixty seconds and delivered a soft-deny into the wrong lane. Capacity blocks imposed a thousand-dollar one-day minimum, and on the hyperscaler enough memory and on-demand did not coexist. Real access came through account escalation, and a neocloud request placed to route around all of it was still cold thirty-six hours later. Scarce accelerated capacity is procured through relationships, not forms. So don't assume bigger hardware removes the constraint. It moves it, from throughput to availability, the same wall the cloud exposes through quota, reservations, and support cases.

Now the honest bounds, because the win here is contained. The Spark is a best case: air-cooled, plug-in, cheap flat amortization, no facility cost. This lab did not measure on-prem at scale, where power, cooling, facilities, and capital re-enter the math at every tier, and whether owned compute still beats the cloud floor there is unmeasured. Tuned-Gemma quality is inferred from saturation, not judged, because the dependency cascade blocked the measurement. Bedrock native fine-tune quality was not measured either; serving sat quota-walled behind a support case. Clean Custom Model Import cold-start latency was never isolated, since the model warmed between attempts, and whether Custom Model Import definitively accepts this model class is unsettled, because the supported-architecture list is a moving patchwork. And the sample is small: thirty questions on a two-judge gate, good for go or no-go, not for fine quality claims. The whole lab cost roughly $1 to $5, since I never secured the rented GPUs to spend on.

So the box is not the production answer. It is the proxy that exposes the constraint. On the box the constraint is token latency, and better hardware bends that curve, but it does not remove the tradeoff. The old fast, cheap, high quality, pick two no longer fits, because availability is now its own axis. The Layer2C version reads: fast, cheap, high-confidence, controlled, and available, pick the constraints you are willing to own. Commodity APIs run out when confidence fails. Managed fine-tunes run out when ownership matters. The box runs out when latency matters. Bigger systems run out when availability and capital matter. Every path runs out of road somewhere, and the lab shows the floor. The question is not who wins. It is which constraint you choose to own.

The numbers

Where each layer belongs

Assessments at the time of the lab

Method and disclosure

Self-funded, no sponsor, free to mix competitors and to tell you not to buy something. The custom-serving economics compare AWS and NVIDIA paths against an owned box, and every path is held to the same measurement.

The quality gate was a frozen 30-question set judged by a cheap strict judge plus one frontier judge, trusted where they agree. Fine-tuning was a local LoRA on the DGX Spark; serving was measured locally with vLLM, on Bedrock Custom Model Import, and against Bedrock native fine-tune pricing. The data plane was held constant on the lab-one AWS S3 Vectors substrate.

The cost structure, the decision table, the access-friction facts, and the architecture pattern ship. The corpus, the taxonomy, the trained judges and adapters, and the specific analytical conclusions stay proprietary.

The essay on this page was rendered from the lab’s frozen structured record by a model writing under this practice’s voice specification, with a deterministic gate checking every figure against the record before publication. The record is the canonical surface: where prose and record disagree, the record rules.