# Borrow the vendor’s plumbing, not its judgment

> Lab 001 · Editorial lab · Status: published · Author: Keith Townsend · June 27, 2026  
> Published by The CTO Advisor LLC (The Advisor Bench) · Layer2C Labs  
> Canonical: https://labs.layer2c.com/labs/spark-s3vectors · Structured: https://labs.layer2c.com/labs/spark-s3vectors.json

**Finding:** The managed data plane is cheap and fast; the cost it hides is the chunking judgment it takes from you. Borrow the plumbing, keep your own chunking.

**Question:** I built a retrieval pipeline across a public-cloud data plane and a local box, the compute I keep below the cloud’s managed abstraction, to map where authority actually sits across the 4+1 stack. The economics were the boring part: eighty-four cents, the cloud faster. The finding worth keeping is what the managed path quietly decides for you, and the two questions the lab now knows to ask.

**Load:** ~237K transcript segments across 5,815 videos, 2013 to 2026, embedded and queried in batch.

**Video:** https://youtu.be/SF1B3C9FASE
**Raw lab detail:** https://labs.layer2c.com/downloads/spark-s3vectors-lab-detail.md
**Vendors:** AWS (cloud), NVIDIA (hw)
**Layers:** Layer 0 (Substrate), Layer 1A (Data), Layer 1B (Retrieval), Layer 2B (Runtime), Layer 2C (Reasoning)
**Themes:** managed-abstraction-cost, designed-partition, authority-placement, ai-factory-economics

## Verdict

**Scope:** The cost verdict is commodity knowledge, and it sits below as evidence. These are the rulings that change how the stack gets scored: where authority sits, and what you cede without noticing.

- **DON'T:** cede the chunking. The managed RAG wrapper re-chunks on its own strategy and turns on parsing you did not ask for, which would have broken label-filtered retrieval. That decision determines what you can retrieve, and it is yours.
- **DO:** borrow the vendor’s plumbing. AWS’s strength is the composable primitives underneath the managed wrapper, not the wrapper. The wrapper is convenience; the primitives are the capability. Score AWS on the primitives.
- **DO:** own Layer 2C as the control structure. The reasoning plane that yields a true answer is the gates, the judges, and the reproducibility around the model, not the weights. Cede the model and you still own the harder half.
- **DO:** place the box by its authority, not its benchmarks. The Spark keeps real local execution authority; its place is development and portability, not commodity inference, where the cloud wins on cost and speed.

**Independence and disclosure:** Self-funded. No vendor paid for this answer.

## Measured

| Measurement | Result | Note |
| --- | --- | --- |
| Corpus | ~237K segments · 5,815 videos | 2013 to 2026; ran the full corpus, not a sample |
| Ingest throughput | 337K segments/hour | full corpus indexed in ~41 min, 0 failures |
| Retrieval latency (from Spark) | p50 280ms · p95 505ms | network-bound; S3 Vectors service ~100ms |
| Generation, local (Gemma 4 26B-A4B) | ~20-23 tok/s | MoE; beats a dense 8B on speed and capability |
| Same model on Bedrock | 136-214 tok/s | 7 to 9x faster than local |
| Setup cost (embed + index) | $0.84 one-time | plus ~$0.07/mo to keep the index |
| Inference cost (full analysis pass) | ~$0.20 | the entire 13-year generation run, at cloud rates |

## Authority placement (DAPM)

**Reading:** Portability: could the enterprise take this layer elsewhere without rebuilding?

| Layer | Placement | Basis |
| --- | --- | --- |
| Layer 0 (Substrate) | Retained | Commodity NVIDIA hardware, so this layer is Retained whoever owns it. Compute runs on a DGX Spark, and the same containers and serving stack are meant to lift unchanged to a bigger platform. |
| Layer 1A (Data) | Ceded | Ceded, because S3 Vectors is AWS’s own vector store with no open exit: the 1024-dimension cosine index and its filterable labels would have to be rebuilt to move. |
| Layer 1B (Retrieval) | Ceded | Ceded, because the embedding model (Titan Text Embeddings v2) and retrieval both run as AWS services with no open exit, so moving means re-embedding and re-indexing. The chunking stayed in the lab’s own code, which is why label-filtered retrieval worked; the managed wrapper would have taken that decision too. |
| Layer 2B (Runtime) | Retained | Retained, because serving runs on vLLM, open software that moves to other hardware without rebuilding. The same model runs faster and cheaper on Bedrock, which is a cost finding, not a placement. |
| Layer 2C (Reasoning) | Retained | Retained, because the gates, judges and reproducibility checks around the model are the lab’s own code and run anywhere. That is the harder half of the reasoning plane, and it stays yours even if the model is rented. |

## What this does not establish

- It did not prove on-prem inference never pays. It proved commodity base-model generation, on this box, for this workload, loses to the cloud on cost and speed.
- It did not settle custom or fine-tuned model economics. Lab two did, and reversed this verdict for custom models.
- It did not prove managed RAG is bad. It proved the wrapper can hide control points, chunking above all.
- It did not test capability against the frontier. An 8B-class model is a cost-structure vehicle, not a frontier contender.

## Open questions

- **What does a specialist data platform add over the hyperscaler’s native Layer 1?** The AWS-native data plane was cheap and did hybrid retrieval in one call. So where does VAST Data, or any performance or portable data platform, earn its keep over S3 Vectors, on latency, on portability, on retained governance? I did not consider that a question worth asking until the bench showed how good the native floor was. Now there is a measured floor to test it against.
- **When does a custom model change the economics?** The cost verdict was measured against commodity per-token pricing, which exists only for base models. The moment a workload needs a fine-tuned model, AWS stops selling tokens and charges a floor instead. Lab two ran that regime and reversed the verdict: against a floor, the owned box wins custom serving above roughly 12 to 20% utilization.

## Objection handled

**But what about utilization?**

The on-prem case is to keep the box busy so the capex pays off. I ran that math. Pegged at 100% for three years, the Spark generates roughly 1.9 billion tokens, which spreads the $4,699 to about $2.50 per million tokens, plus around $0.40 in power. Call it ~$2.90 per million, fully utilized. The same model on Bedrock is about $0.40 per million. So even maxed out for three years straight, the owned box costs roughly seven times more per commodity token.

Utilization is the wrong lever. The box is memory-bandwidth-bound at about 20 tokens a second, so it produces too few tokens an hour for the capex to ever spread thin enough to win. And to hold 70 to 90% you would need a constant firehose of batch work feeding a box that loses even when fed. You cannot util your way past a throughput ceiling.

## Assessments at the time of the lab

| Vendor | Layer | Grade | As assessed |
| --- | --- | --- | --- |
| AWS AI Infrastructure | Layer 1A · Storage | Delegated | June 29, 2026 |
| AWS AI Infrastructure | Layer 1B · Retrieval | Delegated | June 29, 2026 |
| NVIDIA AI Platform | Layer 0 · Compute | NVIDIA Strength — Silicon Authority | May 22, 2026 |
| NVIDIA AI Platform | Layer 2B · Runtime | NVIDIA Authority — Inference + Agent Runtime | May 22, 2026 |
| NVIDIA AI Platform | Layer 2C · Reasoning | Runtime Governance Only — Not a Reasoning Plane | May 22, 2026 |

## Summary

I paired the NVIDIA DGX Spark with Amazon S3 and S3 Vectors as the data plane, in the AWS-managed configuration. Validated on a 5,000-segment subset, then run across the full 13-year corpus, roughly 237K segments and 5,815 videos. The verdict is earned, not projected.

The data plane is cheap, fast, and better than expected. The full corpus embeds and indexes for about $0.84 one-time in roughly 41 minutes, and S3 Vectors turned out to do hybrid retrieval, semantic similarity plus metadata filtering in one call, which is what made label-scoped retrieval work. Retrieval was never the constraint. And the architecture itself held: the 4+1 seams fell where the model predicted, and the public-cloud-to-edge I/O across them was never the bottleneck.

## Detail

The clean architecture is the one where the vendor boundary respects the layer boundary. Embedding is a Layer 1B function, data preparation, even though it runs a model on a GPU. So the whole data plane goes to AWS, and NVIDIA keeps Layer 0 and Layer 2. Each vendor owns whole layers, authority places cleanly, nothing is split. Embedding locally would fracture Layer 1 for no gain, since the corpus is public.

A wrinkle at the bench taught the sharpest plumbing lesson. The fully managed path, Bedrock Knowledge Bases, would not create through the API even for an administrator account, so the same Titan-into-S3-Vectors substrate was self-orchestrated with direct calls. The real loss in the managed path was not orchestration, it was the chunking knob: managed re-chunks with its own strategy and turns on parsing you did not ask for. Bring-your-own kept one segment as one vector with its labels attached, and that is what made hybrid label-filtered retrieval possible. Borrow the vendor’s judgment on plumbing. Keep your own on chunking.

Then the numbers reordered the assumptions. The data plane is cheap and fast. Retrieval was never the problem; the latency seen from the Spark was its own network path, not S3 Vectors. The constraint is local generation.

The architecture itself passed, which is the result the economics can obscure. The 4+1 seams fell where the model said they would. Splitting the stack with the data plane in the public cloud and the reasoning at the edge is an operable partition, not just a diagram: the I/O across that seam, retrieved context down to the box and queries up, was never the bottleneck. The constraint lived inside Layer 2, in local generation throughput, not at the boundary between layers. The designed partition holds.

The box, by contrast, did real work, and that is the fair way to say it. It built and ran the whole pipeline end to end. What it isn’t, at this size, is a match for the public cloud on price for performance. The same model runs 7 to 9 times faster on Bedrock for less money, and the project’s entire local generation would have cost about twenty cents in the cloud. So the GB10 is not the commodity inference tier. But the box doing real work, and the architecture holding, are the results that travel. The same containers and serving stack are meant to lift unchanged to a bigger platform, so the honest close is not that the box failed. It is that this box, at this size, is not price-competitive against commodity per-token pricing, which exists only for base models. Lab two took the custom-model regime, where that price disappears and AWS charges a floor instead, and reversed the verdict: against a floor, the owned box wins. One model-class note worth keeping: a mixture-of-experts model with ~4B active parameters beat a dense 8B on both speed and capability on this hardware, because single-stream decode is memory-bandwidth-bound and active parameters are what move.

There is a second axis the throughput numbers hide. Faster is not better. The mixture-of-experts model won single-stream speed, but a dense Gemma 4 31B reasoned better over the same data, and stepping up to foundation models reasoned better still. That moves the economics from cost-per-token to value-per-answer: a more capable model costs more per token and can still be the cheaper choice in net, when its output carries more business value than the marginal cost. Cheap and fast minimizes the token bill. It does not maximize the worth of the answer. So the open question is not only how cheap the tokens are, but how much capability, measured in value, not throughput.

The transferable lesson runs past the substrate. Generating LLM findings is free and easy. Generating true ones is not. It takes a quantitative gate, cross-run reproducibility, and a strong judge, and judge strength dominates: a mid-tier judge will rubber-stamp a confidently wrong pattern that a strong one refutes. The fix is not the most expensive judge. A cheap strict judge plus one frontier judge, trusted where they agree, did the work. Most naive discoveries did not survive that gate.

## How it abstracts

The shape generalizes. A Layer 1 data service paired with a Layer 2 reasoning surface, testing where the seam falls. Swap the data plane or the reasoning surface and the motion repeats.

So does the verdict. At these corpus sizes, owned-edge inference rarely beats cloud on cost or speed, because cloud inference is already cheap. The case for the local box is air-gap, residency, or as a development instrument that ports to the cluster, not edge-inference economics. The workload lifts to any large-corpus, latency-insensitive task: legal discovery, support ticket mining, research-paper analysis, log and event mining.

## Method and disclosure

Self-funded, with no sponsor, so the lab is free to mix competitors. That cross-vendor mix is the editorial signature, nobody here is selling you one box. Every lab, editorial or sponsored, is held to the same method and the same editorial control; sponsored labs simply center on the sponsor’s architecture.

Validated on a 5,000-segment subset, then run across the full corpus plus a 2026 refresh. Generation used Gemma 4 26B-A4B via vLLM on the DGX Spark, chosen over a benchmarked Llama 3.1 8B and a dense Gemma 4 31B. The data plane used Bedrock Titan Text Embeddings v2 into a 1024-dimension cosine S3 Vectors index, with labels kept filterable for hybrid retrieval.

Substrate engineering and the corpus findings are captured as working lab notes, not shipped. The classification taxonomy, the tagged corpus, the trained judges, and the specific conclusions stay proprietary. What ships is the pattern and the substrate verdict, enough to recognize and abstract.

Placement note restated, September 28, 2026. The Layer 0 row previously gave ownership or location as its reason. The placement stands; the note now gives the instrument’s reason, because the instrument places Layer 0 by the hardware and the runtime adopted, not by who owns or rents the capacity: commodity x86 and NVIDIA hardware is Retained, and elsewhere an open, standard runtime is Retained while a proprietary runtime that ties the workload to one vendor’s hardware is Ceded.

Placement table re-scored, September 28, 2026. Every row is now read for portability, and the table says so at the top. Earlier tables across the labs mixed that reading with the other one, and some rows rested on location, ownership or cost. No placement changed here; the notes now give each row’s reason under that reading.

Layer 1A relabeled, October 16, 2026. The 4+1 canon split Layer 0 into three graded sub-layers, 0A Compute, 0B Network Fabric and 0C Storage, and renamed Layer 1A from Data Storage & Governance to Data Foundation & Governance. Storage is graded at 0C now, so this table’s 1A row sits under a name that no longer covers it. The row scored S3 Vectors, AWS’s managed vector index, and under the split a vector index is retrieval, which is 1B. The placement doesn’t change: S3 Vectors is Ceded either way, because the index and its labels would have to be rebuilt to move, and 1B already reads Ceded here. The Layer 0 row keeps its original citation. It tested compute on a DGX Spark, which is 0A. The rows are left as published.

---
*Layer2C Labs · The CTO Advisor LLC (The Advisor Bench) · labs.layer2c.com*
