Codebase Gardening
Agents make most of the changes to my site and copy whatever they find in it. A month of weekly reviews showed what tending that takes: pull the weeds, then put rules into the code that keep them out.
Some code on my site did nothing at all. A review round found it in late October. An earlier round had found the same kind of code and deleted it. Now there was more.
Agents make most of the changes here, and I approve each one before it goes in. Through October and into November, they also reviewed the whole site every week or so. The rounds found well over a hundred problems, and the agents fixed most of them on the spot.
In a talk called “All Code Is Slop (Except Mine),” Lauren Tan, whose team builds Grok Bot, says “your codebase is kind of like a garden.” Agents copy whatever is growing in it, so you keep it clean “so that agents are always copying the good stuff.”
So the review rounds had been weeding. The weeds kept growing back.
Whatever is in the code gets copied
An agent starting a change reads the code around it first. Whatever it finds there looks like the way things are done here, so it does more of the same.
The agents copied a good choice. A short comment beside the hover styles said every hover effect works one way, and the hover effects they added later followed it.
A choice nobody had made spread just as well. A setting that hid where visitors came from rode along on every new outside link for about eight months.
A broken one spread too. The range of characters that marks Chinese text was copied onto two new pages with a look-alike of its first character, 豈. It looked the same on screen, but it took in Korean too, and it sat there for two weeks.
Code has always taught whoever reads it next, so a messy codebase always cost something. A person picks up a codebase slowly and usually learns why things are the way they are, while an agent starts fresh every time and writes fast. The look-alike range reached two new pages in three days, and on Lauren’s team agents send in thousands of changes a month. So a weed now spreads at the speed the agents write.
What a month of weeding turned up
The rounds pulled three kinds of weeds. A few were copies of something already in the code. About three times as many were new, left behind by the agents’ later work. Most of the rest were old weeds that had been there all along. An earlier round had pulled the one in front of it and missed the others. A few came from outside the code, like links that went dead. Two agents sorted every repeat separately, and both landed on roughly these shares.
So copying explains how a weed spreads more than how it gets in. The link setting spread for eight months, and every leftover left in place is something the next agent could copy. Lauren’s talk puts the weeding in those terms: “cleaning up old examples is part of improving future output.”
The unused code from the start of this post is the clearest new weed. It turned up in four separate rounds, and after the first round most of it was new. In a small game on the site it was extra code for old browsers. Elsewhere it was styles for things that had been removed, and settings nothing read.
Comments grew just as fast. In two weeks of October, comment lines on the site more than doubled while the code grew by about two-thirds. The agents explained their work as they went. Then the code changed, and nobody updated the explanations. One comment went out of date within a day, when a later change rewrote the test it described. In one cleanup, an agent shortened a batch of comments, and one of the new versions said the opposite of what its code did. Lauren’s team bans about 99% of comments for this reason. Comments go stale, and a comment that explains a workaround makes it “look intentional to the next agent.”
Pulling weeds one at a time gets the weed a round finds. It doesn’t stop the next one from growing, and it doesn’t reach the old ones nobody has looked at yet. Each fix came with a short write-up of what went wrong and why. The write-up stays with that change, and the next agent starts from the code. On the Lightcone podcast, Lauren called the code base “a form of memory.” It’s the memory every agent reads before a change.
Rules that keep the weeds out
So after weeding, the rules have to go into the code itself. Lauren’s talk gives an order for each one. If an instruction for the agents can become a check, make it a check, and if a change in design can take the wrong choice away, that beats the check.
On the podcast she put it in terms of time. She spends most of her effort on the code base and its checks, and the least on human review. The hours that would go into review get “encoded into your code base” instead. So a fix belongs as close to the code as it can go, where nobody has to remember it.
The strongest rule is a design that takes the wrong choice away. Lauren’s team keeps one React feature out of product code entirely, because it takes a lot of nuance to use well and agents aren’t good at it. That mistake can’t happen there anymore. On my site, the rule for outside links moved into one shared piece of code. When I changed the rule later, the change took one line.
Next comes a check that runs on every change and can stop it. On the podcast she also described one her team built. It keeps the part of their app that draws the window from using code from the part that does the heavy work, because slow code there makes the window stutter. When a mistake keeps coming back, her team writes a lint error that points the agent to the real fix. My site already had a check that could have stopped some of the unused code. The type checker reads the site’s code before each build, and it flagged the game’s old-browser code the day it arrived. But it only printed a hint, so the change went in, and the code sat there for three days until a review removed it.
Then come written rules, and they hold best right next to the code. The hover comment sat beside the shared hover styles, and the changes there followed it. Along with the big comment cleanup, I added a rule to the file my agents read before they start: keep a comment only when it guards against a real mistake. Comments have stayed about where the cleanup left them. One had already gone stale the same day, and a review caught it two days later.
About one in five problems got a design fix and about one in seven got a check. None of the cases those fixes covered has come back so far. Similar ones turned up in code the fixes never reached, and review handles the rest.
A shared place covers the code that uses it. Changing the link rule also turned up a link someone had typed by hand, which had never gone through the shared code. Links inside posts hadn’t either. They kept opening in the same tab for eight more days, until a later change gave them the rule. And no check can tell whether a comment still says something true. That part is still up to review.
The copy that was still there
While I was writing this, an agent checked the Chinese range again. The October cleanup had moved it into one shared place, and the shared copy was right. The song page still had the look-alike in two places, in the code that works out how many lines a long title takes. Like the hand-typed link, that code had never used the shared range.
I made two changes after that.
The type checker’s hints now stop a change from going in.
The song page uses the shared range, written as numbers, so a look-alike would show up as a different number. One test checks the range with 豈 and with Korean. Another fails if any other file writes its own range.
Unused styles and stale comments are still outside what these checks read. They wait for the next round of weeding.