June 8, 2026
Readme or red flag?
Do agents.md files help coding agents?
Turns out the magic instruction file may be slowing the robot coder down
TLDR: A study found that giving AI coders extra project instruction files often didn’t help much — and sometimes made them slower and more expensive. Commenters split hard between “the notes are useless superstition” and “you’re just writing bad notes,” which matters because more teams are starting to rely on these AI helpers.
A fresh research paper asked a very internet-age question: if you leave a little instruction note for an AI coding helper — basically a file telling it how a project works — does it actually do better? The awkward answer: not really. In the tests, AI-written instruction files were slightly worse or barely different from having no file at all, and the biggest shocker was that using no file was often cheaper and faster. Developer-written notes did beat AI-generated ones, but the big headline for the crowd was the same: the “helpful guide” may be turning into homework.
And oh, the comments came in hot. One camp basically yelled, “This proves nothing except that you’re stuffing the wrong info in there,” with one user insisting that linking to spec docs saves them from the classic “confused new robot dropped into your codebase” problem. Another camp was far less polite. One commenter absolutely torched the whole ritual, comparing hypey instructions in AGENTS.md to a shaman burning a bone and calling the practice glorified superstition. Brutal.
The funniest mini-drama? People got weirdly specific. One commenter swore shorter notes are better and claimed bizarrely precise wording like “88.47%” works better than “100%,” which is exactly the kind of oddly confident internet advice that readers love and fear. Others pointed out the results may already be aging badly because newer AI models might behave differently. So the community verdict is messy, spicy, and very on-brand: good notes from real humans might help, generic AI fluff probably doesn’t, and nobody trusts the benchmark without arguing about the setup first.
Key Points
- •The article reviews a paper that evaluates whether repository-level instruction files such as AGENTS.md and CLAUDE.md help coding agents.
- •The study uses two evaluation settings: SWE-bench Lite and a new benchmark called AGENTBENCH.
- •AGENTBENCH contains 138 Python tasks from 12 repositories with developer-provided context files.
- •In the reported results, LLM-generated context files slightly reduce task success or make little average difference compared with using no context files.
- •Trace analysis found agents generally follow context-file instructions, but they also perform more searches, file reads, tests, and tool usage, while no-context runs were cheaper and more efficient.