Earlier quoted context omitted.
> Document what's there, not the diff We recently added a similar thing to our style guide, It’s astonishing to me that we have to spell this out, that something as obvious as this needs to be explained to LLM’s at all. They’re supposed to be exceeding human intelligence, at least at things like programming, but can’t understand basic things like what code comments are.
My theory (which might be completely wrong) is that models do this because it improves quality for vibe coders. When vibe coding the content of user prompts is ground truth and the only way any human thought affect the code base. So if the vibe coder says "do X not Y", recording int comments that "we shouldn't do Y" is important. It ensures that the agent doesn't accidentally decide to do Y tomorrow, which would frus…
A recent example I saw was an agent leaving a comment "// return an error here instead of panicking, as a panic will abort the process". Because likely the original human in the loop caught the AI putting a panic in there and told them not to, and then the comment to not do panics was placed in there. But to a future agent, it'll see that and think "ok, this comment must be here because we usually do use panics instead of returning errors, this place must be an exception", and now its context window is primed to think of using panics first.
It's pretty well-documented at this point that spending a lot of tokens explaining what not to do can actually increase the likelihood of an LLM doing that thing, especially when its context window is getting full.
It's like if you go to the grocery store and see a sign saying "Vegan tomatoes". It sounds fine until you think "wait, aren't all tomatoes vegan?" and now you start doubting yourself and start imagining what a non-vegan tomato would be.
Another semi-related issue is the tendency for LLM's to make up their own dumb little short-hand words for things that it has been talking about over and over in the context window. "The lock that prevents a user from being deleted while another thread is updating it" becomes a "user-fence", and now there's comments saying "// this function returns a user-fence", and I have absolutely no idea what that's supposed to even mean.