Hi, I'm Samyak.
I build AI products, and I write about the parts that are still hard: getting agents to finish long tasks, deciding what to build on a model that keeps changing, and working out what to charge for software that does the work.
The half-life of an agent
Why an agent that looks finished in a demo fails in production, worked out in arithmetic, and what the arithmetic says about what to ship.
Scaffolds and complements
The model under your product gets better and cheaper every few months without your help. Some of what you build loses value each time it does. Some of it gains. Telling the two apart is most of roadmap planning now.
The reader that never skims
Your product has a second user now. It reads every word, pays for each one, takes instructions literally, and acts on someone else's behalf. Designing for it is a different craft.
Four cents a copy
When software does the work itself, charging per seat stops making sense. What replaces it depends on a question from contract theory, namely who should carry the risk when the work goes wrong.
Case law for model behaviour
A research team asks you to make the model better at science. Before anyone can build an eval, someone has to say what better means, and the people who know best disagree. A method that treats their disagreement as the raw material for the spec.
Busy signals
Flat subscriptions get wrecked by a small number of very heavy users. Meters and caps punish the best customers. A pricing design built on one observation, that the runaway usage comes from work nobody is watching.
The diagonal
Acceptance rates reward rubber-stamping, time-in-product falls when an agent works, and surveys measure feelings. A metric that compares how much people rely on an agent with how much it has earned, task by task.
The 93% problem
People approve almost everything an agent asks permission for, so the approval step protects very little. How to design human oversight whose miss rate you can measure, using a technique from airport security.
The smallest failing trace
An agent fails one run in twenty. The trace is eighty steps long, and when the developer runs it again it works. Why tracing dashboards do not solve this, and how a platform could shrink every failure to the few things that cause it.
Hyrum's law for models
The new model scores higher on every eval, and users say the product got worse. Why average scores are the wrong test for an upgrade, and a launch gate built on counting what broke.
Experience
- Expedia GroupAI product, agents
- Canaan PartnersVenture investing
- AdobeEngineer, then product manager
Education
- Kellogg School of ManagementMBA, Artificial Intelligence
- NSIT, University of DelhiB.E., Electronics
Finishing the MBA in Artificial Intelligence at Kellogg, and working with a large retailer on a capstone that tests when a small fine-tuned model can stand in for a large hosted one.