Spec-driven, model-tiered AI development
How I get reliable output from unreliable components: treat the spec as the durable artifact, put cheap models in charge of process and expensive models in charge of judgment, and never let a builder grade its own work.
The spec is the durable artifact
In agentic development, code is cheap to regenerate and conversations evaporate. What survives -- what is worth an expensive model's time to write and a human's time to review -- is the spec. I write specs the way a PM writes requirements: the problem, the constraints, the acceptance criteria, and the tunable surface (anything a human might want to change later lives in config from version one, never as a retrofit).
A good spec makes the implementing model interchangeable. When a build goes sideways, I do not patch the conversation; I fix the spec and regenerate. Big refactors get a fresh spec rather than a maintained pile of amendments -- superseding beats archaeology.
Cheap models host, expensive models think
Most of an agentic workflow is not thinking. It is dispatching: watch for a trigger, launch a worker, collect a result, post a status. I run that layer on small, cheap models with mechanical instructions -- string triggers and fixed verbs, no judgment calls. Judgment is concentrated at checkpoints, where a frontier model reviews a spec, audits a diff, or makes a taste call.
- Host tier -- small models run loops, route messages, and follow checklists. If a step needs discretion, the design is wrong.
- Build tier -- mid-size models implement against the spec. They are fast, cheap, and honest about being replaceable.
- Judgment tier -- the expensive model writes the spec, reviews the result, and owns the hard calls. It shows up briefly and leaves artifacts behind.
This is a budgeting discipline as much as an engineering one: frontier-model tokens go where they change the outcome, not where a while-loop would do.
Cold verification gates
A model that just built something is the least trustworthy judge of it. So verification runs cold: a fresh context, a different model or tool, no access to the builder's rationalizations. Clone the repo and build it. Curl the endpoint. Run the evals. The gate either passes on observable behavior or it fails -- prose explanations are not evidence.
The same principle covers humans: anything destructive, outward-facing, or money-spending stops at a human-in-the-loop approval gate, by architecture rather than by convention. Guardrails you have to remember are not guardrails.
A working public example
underthink, a Claude Code plugin, is the small public end of this methodology -- built spec-first, with the model-tiering philosophy applied to the tooling itself. The larger private example is the six-machine home agentic network, where the same tiers and gates govern a fleet that non-technical people rely on daily.
What this looks like in practice
| Phase | Who does it | Artifact |
|---|---|---|
| Frame the problem | Human + frontier model | Spec with acceptance criteria |
| Implement | Mid-tier models, in parallel lanes | Code, config surface, tests |
| Verify | Cold context: fresh model, real commands | Pass/fail evidence, not prose |
| Approve | Human, at the gate | The decision, logged |
The pattern scales down to a one-file utility and up to a multi-machine agentic system. The constant is that every layer produces something inspectable -- a spec, a diff, a gate result -- so trust accumulates in artifacts instead of vibes.