The diagonal
A proposal for a success metric for agent products, built on the idea that the goal is the right amount of trust for each task.
The problem
Every product team needs a number that tells it whether the product is getting better. For most software the candidates are familiar: how many people use it, how long they stay, how often they come back.
Agent products break each of these.
Time in product goes the wrong way. A good agent finishes the job and the person leaves. A team that drives this number up is making its product slower.
Acceptance rate, the share of the agent’s suggestions that people accept, looks more promising. But Anthropic has reported that users of its coding agent accept 93% of permission prompts, and describes this as approval fatigue. A high acceptance rate cannot distinguish a product people rightly rely on from one they have stopped checking.
Task completion measures the agent. It says nothing about whether people use it for the right things.
Trust surveys measure how people feel. The harder issue is that more trust is not always the goal.
That last point is the heart of the problem. In 2004 John Lee and Katrina See published what became the standard reference on the subject, Trust in Automation: Designing for Appropriate Reliance. Their argument is in the title. Trust that exceeds what a system can do leads people to misuse it. Trust that falls short leads them to ignore it. Both are failures, and a single number that goes up with trust will reward one of them.
So the hard version is this: what metric improves only when people rely on an agent as much as it deserves, and no more?
What forecasters worked out
Weather forecasting had the same problem with its own product. A forecaster who says “70% chance of rain” cannot be judged right or wrong on any one day.
The solution was to judge them over many days. Collect every day on which the forecast was 70%, and count how often it rained. If it rained on about 70% of those days, the forecaster is calibrated. In a well-known 1977 study, Allan Murphy and Robert Winkler found that American forecasters were close to this ideal: their stated probabilities of rain matched how often it rained.
The standard picture of this is a chart with the stated probability on one axis and the observed frequency on the other. A perfectly calibrated forecaster sits on the diagonal line. Nobody asks whether the forecaster is confident enough. They ask how far the forecaster is from the diagonal.
The proposal: a calibration chart for reliance
I propose applying the same picture to an agent product, with one change. The forecaster’s chart compares stated confidence with outcomes. This one compares how much people rely on the agent with how reliable it is.
Divide the agent’s work into task types. For each type, measure two things.
- Reliability. How often the agent gets this kind of task right, measured by auditing a sample of results, including the ones users accepted.
- Reliance. How often users accept the result without checking it.
Then plot one against the other.
Reliance against reliability, by task type
Illustrative example for a customer-support agent
Three things show up on this chart that no single metric reveals.
Tasks above the line are over-trusted. In the example, people accept refunds without looking 93% of the time, though the agent gets them right only 72% of the time. This is where harm comes from.
Tasks below the line are under-trusted. The agent summarises threads correctly 97% of the time, yet people re-read the whole thread anyway. Nothing bad happens. The product is simply delivering half the value it could.
And the spread matters. A product whose points are scattered far from the line in both directions has users who cannot tell what the agent is good at.
The metric that follows is the average distance from the diagonal, weighted by how common each task is. I would report the two sides separately, because they cost different things.
The two sides call for opposite work
| Over-trusted task | Under-trusted task | |
|---|---|---|
| What it costs | Errors reach customers because nobody looked | People redo by hand what the agent does well |
| Who notices | Nobody, until something goes wrong | Nobody, ever |
| What to build | Show the evidence behind the answer; require a look before the action runs; show the agent's record on this kind of task | Show the track record; make checking take seconds; offer to handle the next one without asking |
| Effect on usage metrics | Acceptance falls, and that is the goal | Acceptance rises |
This is the part I find most useful. The chart tells a team to build features that reduce usage on some tasks. Under any engagement metric that work looks like a regression, and it would be cut. Measured against the diagonal it counts as progress.
Measuring reliance without asking
Reliance has to be read from behaviour. Surveys are too slow and too flattering.
Behavioural signals of reliance
| Signal | What to log | What it says about reliance |
|---|---|---|
| Review depth | Whether the person opened the result, and how long they spent relative to its length | Accepting a long change after two seconds is reliance without checking |
| Edits before accepting | How much of the output was changed | Heavy editing means the person is using the agent for a first draft and no more |
| Later reversal | Undo, revert or redo of the same task within a day | The best cheap evidence that an accepted result was wrong |
| Interruptions | How often the person stops the agent mid-task | Rises with experience; a sign of closer attention |
| Delegation size | How large a task the person hands over in one go | Grows as trust grows, and shows which tasks it is growing on |
Anthropic’s 2026 study of how people actually use its coding agent shows why these signals need to be read together. New users let the agent run without approval in about 20% of sessions. By the time they have several hundred sessions behind them, that figure is over 40%. Yet experienced users also interrupt the agent more often, in about 9% of turns against 5% for newcomers.
On a single trust scale those two facts contradict each other. On the chart they make sense. Experienced users have learned which tasks deserve reliance and which deserve a close watch. They delegate more and step in more precisely. Their points sit closer to the diagonal. Lee and See called this resolution: the ability to tell situations apart. It may be the best sign that a product is teaching its users well.
The same study describes a “deployment overhang,” where the autonomy models can handle exceeds what people give them in practice. In the terms of the chart, that is a product with most of its weight below the line.
How to tell if it is working
- Distance from the diagonal, split into the over-trust part and the under-trust part. Both should shrink.
- Slope by experience. Plot the chart separately for new and experienced users. Experienced users should track reliability more closely.
- Harm from accepted results. Errors found in audited samples of work that users accepted unchecked. This should fall as the over-trust side closes.
- Correct work per hour of the user’s time. This guards against a team gaming the metric by making people check everything.
Where it could go wrong
Reliability is expensive to measure. Someone has to audit a sample of outcomes for each task type, and keep doing it as the model changes. Without that, the horizontal axis is a guess. The chart does not fix this cost. I think it justifies it, since the audit is also what tells you how good the product is.
Task types can be drawn badly. If “issue a refund” mixes easy and hard cases, its single point hides an over-trusted group inside an average. When a task type has a wide spread of outcomes, split it.
The metric can be gamed downward. A team could reach the diagonal by adding so much friction that people check everything. That is why it must be paired with a measure of value delivered.
Reliance lags reliability. After a model upgrade the agent may be better at a task for weeks before users notice. The gap that opens below the line after each upgrade is a measure of how well the product communicates what has changed.
The forecasters’ lesson was that confidence should be judged against a record of outcomes. An agent product can hold its users’ reliance to the same standard. The team’s job is then to move each task toward the line, from whichever side it is on.
Sources
- Lee and See, Trust in Automation: Designing for Appropriate Reliance, Human Factors, 2004.
- Murphy and Winkler, Reliability of Subjective Probability Forecasts of Precipitation and Temperature, Journal of the Royal Statistical Society, Series C, 1977.
- Anthropic, Measuring AI agent autonomy in practice, 2026.
- Anthropic, Claude Code auto mode, 2026.