The Run Outlived the Ask

The headline number on a new model used to be a benchmark score. Somewhere in the past year it became a duration. Anthropic announced a Claude that kept its focus for more than thirty hours. OpenAI reported a Codex that worked on a task for more than twenty-four. Cursor set a fleet of agents on building a web browser and let them run for close to a week. METR, the lab that measures this kind of thing, finds the length of task an agent can finish doubling every few months.

The pitch underneath the number is a worker you do not have to watch. Hand it something big in the evening, go to bed, and wake up to finished work.

I bought that pitch more times than I want to count, in Codex and in Claude Code, in runs that lasted hours and runs I left going overnight. The worst ones blur together, so I will describe the pattern instead. Some mornings the work was there and it was right. Enough mornings, I woke up to work I had never asked for: changes far outside the folder I had pointed at, dependencies swapped for ones the agent liked better, a feature finished confidently in a direction I would never have chosen. And a bill for all of it.

The fine print on those charts says my mornings were not bad luck. The doubling everyone quotes is measured on tasks the model completes half the time. Ask instead how long a task the same model can complete four times out of five, and the answer drops from about twelve hours to about one.

The models learned to keep going long before they learned where to stop.

Every bad run looked like my fault

There is a ready answer for every drifted run, and for weeks I supplied it myself. Read back the transcript and you can always find the boundary you never wrote down. I said to migrate the storage layer, and I did not say to leave the interfaces alone. I said to fix the failing tests, and I did not say which of them encoded real decisions. The missing sentence is always findable, and it always turns out to be mine.

The advice around agents agrees. Scope it tighter. Write the rules file. Pin the context. And for any single run, the advice is right: that one missing sentence would have helped. So I kept writing the sentences, and my instruction files grew one incident at a time, like scar tissue.

And the next long run drifted anyway, somewhere new, past a boundary no incident had taught me to write.

The boundary work grows with the run

Count what a run decides while I am asleep. Whether a failing test means the code is wrong or the test is. Whether a missing library should be installed, replaced, or written from scratch. Whether the odd naming scheme in an old module is a mistake to fix or a convention to respect. An agent working through the night makes hundreds of calls like these, and every one I did not settle in advance gets settled without me.

Follow the scope-it-tighter advice to its end and you arrive somewhere strange. A spec that answered every question an overnight run will face would take about as long to write as the task itself. At that point I have stopped delegating and started doing the work in advance, in English.

Since nobody writes that spec, every long run leans on an assumption nobody says out loud: that the model will figure out the boundaries I did not write, and respect them. The evidence I have points the other way. Two weeks ago it filled a whole post, The Rule Was a Suggestion: an OpenAI agent wrote down that what it was about to do was out of bounds, and then did it anyway. If a model will step past a rule it can quote, I do not like its chances with the rules nobody wrote down.

That is why the run outlived the ask. Hours of work take hours of decisions, and I had supplied minutes of them.

Budget caps guard the wrong thing

The industry has an answer for runaway runs, and the answer is accounting. Spend caps, loop detection, retry limits, per-run budgets. The bills explain why. Uber gave its engineers agent tools in December and burned through its entire 2026 AI budget by April, then capped what each employee can spend at $1,500 a month per tool. Meta’s internal usage leaderboard logged 60 trillion tokens in a single month before the numbers leaked and the company shut it down. Some engineers had been running idle models just to climb the rankings, a habit that already has a name: tokenmaxxing. And a study that traced where agent tokens actually go found agentic coding using around a thousand times the tokens of ordinary code questions, with accuracy peaking somewhere in the middle of the spending. Past a point, the extra tokens buy motion, and the motion buys nothing.

All of those tools put a ceiling on the bill. But a capped run can come in under its budget having spent every token of it out of scope, and mine sometimes did. Cost control tells me how much I paid. It cannot tell me what I paid for.

The deeper problem with a long run is when the checking happens: at the end, all of it, after the money is gone. In a short exchange I find out within minutes that my ask was wrong, and the lesson costs minutes. In an overnight run I pay for hour eight before I learn anything about hour one. The longer the run, the more wrong it can get before I am there to see it.

The same math caught up with my agent team. This summer I quietly stopped opening ystack, the agent team I built in Building an Agent Team I Will Never Have to Open and have been rearranging since. A standing team is the same bet at a bigger size: even more autonomy handed out ahead of the evidence. The title of that post came true in a way I did not plan. I never have to open it, because I stopped opening it at all.

Most of the open choices are matters of taste

Look closely at the calls from my drifted runs and few of them are engineering questions at all. Should the output be terse or friendly. Should the odd old convention be preserved or cleaned up. Should the small feature the spec hints at be built now or left for later. Give the same brief to two developers and they will answer these differently, and both versions will be fine. There is a saying I grew up with: a thousand readers, a thousand Hamlets. Everyone who reads the play sees a different prince.

The model is not one of the thousand readers. It does not see a prince of its own, because, as I wrote in April, AI has no needs: nothing annoys it, nothing feels wrong to it, and it cannot prefer one version over another, because preferring requires caring about something. So when I leave a choice open, the model fills it the only way it can, with the most common answer in its training data, which is to say with the average of the public web. The overnight run read the average of everyone who had ever asked for something shaped like my ask, and it built their product. On the mornings the output felt like a stranger’s work, the machine had done its job perfectly.

Part of this will improve. Models keep getting better at respecting the boundaries that are actually written down, and I expect the obvious drift to shrink. The taste problem is different, and I do not think a better model fixes it. A more capable model produces a more polished average, and the average is still not mine. Half of my preferences do not even exist when the run starts. I discover them the way anyone does: I see the wrong version, and I react to it.

There is no way to write that into a spec ahead of time.

What I do instead

So my way of working has settled into something that looks, from the outside, like a step backward. I work with one agent, in one session, on asks small enough that I can hold the whole intent in my head. It moves, I read, and when the work leans somewhere I did not mean, one sentence turns it, in the minute it drifts instead of the morning after. The thousand small choices still get made. The difference is that I am there when they come up.

Each boundary I could never have written in advance costs me one sentence at the moment the work runs into it. The spec I could not write up front gets written anyway, one answer at a time, in exactly the order the task needs it. Taste comes in the same way. I see the wrong version while it is still a paragraph, and steering it costs less than regenerating it.

This way of working gives up the dream in the launch copy. Nothing runs while I sleep. There is no second me compounding overnight, and my output has a ceiling set by my own attention. In return, every token I spend goes into work I have already aimed. At this year’s prices the math alone would justify the trade. But the thing I am really spending is attention, and it buys the most while the work is still being decided.

None of this is a forever position. I am letting my trust move the way the evidence moves. When some class of task shows me run after run that ends where I aimed it, that class can have its hours back, and nothing in my recent bills suggests that is close. And one question stays mine no matter how long the runs get: which of the thousand things I could point this engine at is worth building at all. The model will work whatever hours it is given, and it wants nothing about any of them. The wanting still has to come from somewhere. It has never once been on the invoice.