New Research Reassesses the Value of Agents.md Files for AI Coding

It's a best practice to document things about the code base, so that other devs (even senior devs) don't start to do things differently. This will probably not change

What I think is short lived is this insistence in separating LLM instructions from general documentation for both humans and AI. LLMs can read human docs, and concerns about context window size will probably disappear

But maybe future docs will be LLM-first, but people won't read them directly. They will ask a LLM questions about it

Except that most READMEs are seemingly written more for end-users than for developers; and even CONTRIBUTING files often mostly just document the social contribution process + guidelines rather than providing any guidance targeted toward those who would contribute. There’s a lot of “top-level architectural assumptions” detail in particular that is left on the floor, documented nowhere. Which “works” when you expect human devs to “stare really hard and ask questions” until they figure out what’s being done differently in this codebase; but doesn’t work at all when an LLM with zero permanent learning capability gets involved.

I find this is very heavily situation dependent. I tried getting even Opus to implement a xero integration using the python sdk, without telling it much about the sdk besides the example code/instructions on github and it made an absolute un-runnable mess out of it.

The sdk and examples appear to be very atypical for a lot of python I've done (and atypical for my code base). Giving it a context/how-to file I put together made it significantly better (however I still gave up and did it more manually).

The trace analysis is the most interesting part of this paper.

Agents dutifully follow AGENTS.md instructions — they run more tests, grep more files, do more checks — but that thoroughness costs 19% more inference without meaningfully improving outcomes. That's not an argument against context files, it's evidence we're writing them wrong.

We're authoring them like READMEs for humans ("here's the architecture, here's how we structure things") when agents actually need narrow, non-inferable directives — the custom build flag, the weird test harness, the constraint they'd waste 30 tool calls discovering.

The HN commenter who noted the real value is "forcing you to articulate things previously just in your head" is actually pointing at the same conclusion from the other direction: the thinking matters, but the artifact needs a different shape than what we've converged on.

[flagged]