11–20 of 259 posts
We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation. Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
pfft...I'm way past understanding
We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation. Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
I am so dying to read more about the new/current/real bottleneck! Where is the bottleneck? WHERE?? Tell me! No evidence needed, just lay it on, man to man, thought-leader to thought-leader!
This is my new chat-up line at networking events.
We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation. Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
Are these generated descriptions of LLM submitted changes or of human changes? If a human, shouldn't they be putting the motivation into the PR?
While the tips are good to handle the volume, I still think this sets code owner on a dangerous path. AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high. The article I would like to read would suggest how to force LLM to architect the code like a solid tower…
The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags.
Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model.
(In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.)
LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.
Earlier quoted context omitted.
Are these generated descriptions of LLM submitted changes or of human changes? If a human, shouldn't they be putting the motivation into the PR?
LLM-generated (or at least LLM-assisted), but a human's still the one submitting the PR for review.