Scaffolds and complements
The model under your product gets better and cheaper every few months without your help. Some of what you build loses value each time it does. Some of it gains. Telling the two apart is most of roadmap planning now.
For about forty years, software teams planned against a curve. Chips doubled in power on a schedule that everyone in the industry knew, so a team starting a two-year project designed for the computers that would exist when it shipped. Software that felt too heavy at the start of the project ran comfortably by the end of it. Planning for hardware that did not exist yet was ordinary practice.
Teams building on language models have a curve too, and it is steeper. Many roadmaps are still written as if the model were a fixed part.
Two curves
The first curve is about ability. The research group METR tracks the length of task, measured in the time a skilled person would take, that the best models can complete half the time. In their 2025 paper that length had doubled about every seven months for six years, and their more recent measurements have run faster than that.
The second curve is about price. Analysts at Andreessen Horowitz found that the price of a fixed level of model capability had been falling about ten times a year. Epoch AI measured the same thing across many tasks and found declines between nine and nine hundred times a year, depending on the level of capability you hold constant.
What three more years of the published trends would mean
Starting from today, if the seven-month doubling and the ten-times-a-year price decline both hold
I would not bet a company on either line holding exactly. I would bet on the direction. A feature that takes two quarters to build will ship into a world where the model can handle tasks roughly twice as long and costs a fraction of what it cost when the work began.
The bitter lesson, for products
In 2019 the computer scientist Rich Sutton wrote a short essay called The Bitter Lesson. Looking back over seventy years of AI research, he saw one pattern repeat in chess, in Go, in speech and in vision. Researchers built their own knowledge of the problem into the system, and it helped for a while. Then a general method with more computing power behind it overtook them, and the hand-built knowledge became dead weight.
The lesson was written for researchers. It applies just as well to product teams, with one change. In research, the thing made obsolete is a technique. In a product, it is a feature, with a team, a maintenance cost, and often a line on a pricing page.
Two kinds of work
When I look at a roadmap for an AI product now, I sort every item into one of two groups.
A scaffold is anything built to make up for something the current model does badly. A complement is anything that a better model makes more valuable.
Scaffolds and complements
| Scaffold | Complement | |
|---|---|---|
| What it is | Something you build because today's model cannot do a thing well | Something the model cannot get by becoming smarter |
| Examples | Splitting documents into chunks, parsers that repair malformed output, long chains of prompts that walk the model through a plan | Your customers' data, permission to act inside their systems, a record of their corrections, an evaluation set that defines "good" for the job |
| When the model improves | Worth less, and sometimes gets in the way | Worth more, because a stronger model does more with it |
| How to build it | Cheaply, in one place, with a written condition for removing it | Carefully, early, and as if you will own it for ten years |
The word complement comes from economics. Two goods are complements when a fall in the price of one raises demand for the other, as cheaper cars raise demand for petrol. Ajay Agrawal, Joshua Gans and Avi Goldfarb applied the idea to AI in their book Prediction Machines: as prediction gets cheap, the things that go with it, such as data, judgment and the ability to act, become more valuable. I find the same reasoning holds one level down, inside a single product.
There is a quick test for which group a feature belongs to. Ask: if the model were twice as capable tomorrow, would this matter more or less?
A pipeline that cuts documents into pieces because the model can only read a few pages at a time matters less. Models that once read a few pages now read whole books. Access to a customer’s booking system matters more, because a more capable model can do more with it. A library of prompts tuned to stop one model from making one kind of mistake matters less. A set of five hundred real cases with agreed right answers matters more, because it is how you will know whether the next model is safe to switch to.
How each kind of work ages
An illustration, not measured data
Scaffolds are still worth building
None of this means scaffolds are a mistake. They are how a product ships this quarter. A model that is almost good enough, plus a scaffold, is a working product. Waiting for the model to improve is a way of handing the market to someone else.
The mistake is building a scaffold as if it were permanent. Three habits help.
Build it cheaply and in one place. A scaffold spread through the codebase is hard to take out. One that sits behind a single interface can be switched off with a flag.
Write its removal condition into the spec. For example: “Remove the retrieval re-ranker when the model alone scores above 90% on the retrieval evaluation set.” This sounds fussy. In practice it is the only way a scaffold ever gets deleted, because nobody otherwise has a reason to check.
Test with the scaffold off at every model release. This is where evaluations do their real work. An evaluation set is usually described as a way to catch regressions. It is also the instrument that tells you which parts of your product the new model has made unnecessary. A release will often make some of that code unnecessary, and the simpler system is easier to run and to reason about.
A roadmap in three lists
Sorting work this way changes what a roadmap looks like. I keep three lists.
- Works today. Features the current model can support, with scaffolding where needed. This is what ships.
- Almost works. Features that fail too often today but are close. Here the right move is to build the complement side now: collect the data, negotiate the permissions, write the evaluation set. When the model catches up, everything else is ready, and the feature ships in weeks.
- Does not work yet. Ideas that failed when tried. Keep the prototypes and re-run them on each new model. The day one of them passes is worth knowing about before your competitors do.
The second list is where I think most of the advantage is. A competitor can adopt the same model on the same day you do. They cannot produce two years of customer corrections or a signed data agreement on that day.
Where the rule bends
Two cautions.
Trends are evidence about the future, and they are weaker evidence than they look on a chart. Plan for a range. If the model improves half as fast as the trend suggests, the “almost works” list takes longer to pay off, and the scaffolds stay in service longer. That is a cheaper error than the opposite one.
And some scaffolds turn out to be permanent for reasons of cost. A small, cheap model with careful scaffolding can beat a large one on margin even when the large one needs no help. In a capstone project with a large retailer, my team is testing exactly this trade: a small fine-tuned model against a large hosted one, and the volume of requests at which the answer changes. The scaffold-or-complement question is about value. Whether the scaffold is still worth running is a question about cost, and the two can have different answers.
Sources
- Kwa et al., Measuring AI Ability to Complete Long Tasks, METR, 2025.
- Guido Appenzeller, LLMflation: LLM inference cost is going down fast, Andreessen Horowitz, 2024.
- Epoch AI, LLM inference price trends, 2025.
- Rich Sutton, The Bitter Lesson, 2019.
- Agrawal, Gans and Goldfarb, Prediction Machines, 2018.