Really nice site design btw
Two things LLM coding agents are still bad at
311–320 of 382 posts
Re: Two things LLM coding agents are still bad at
#312Earlier quoted context omitted.
The last point I think is most important: "very subtle and silently introduced mistakes" -- LLMs may be able to complete many tasks as well (or better) than humans, but that doesn't mean they complete them the same way, and that's critically important when considering failure modes. In particular, code review is one layer of the conventional swiss cheese model of preventing bugs, but code review becomes much less eff…
I think we need better code review tools in the age of LLMs - not just sticking another LLM to do a code review on top of the PR Needs to clearly handle the large diffs they produce - anyone have any ideas
I was about to write my own tool for this but then I discovered:
git diff --color-moved=dimmed-zebra
Re: Two things LLM coding agents are still bad at
#313Earlier quoted context omitted.
> I don't think they were ever really sold as that, and we have better tools for that. We have OpenAI calling gpt5 as having PhD level of intelligence and others like Anthropoc saying it will write all our code within months. Some are claiming it’s already writing 70%. I say they are being sold as a magical do everything tool.
They are lying, because their salary depends on them lying about it. Why does it even matter what they're saying? Why don't we listen to scientists, researchers, practicioners and the real users of the technology and stop repeating what the CEOs are saying? The things they're saying are technically correct, the best kind of correct. The models beat human PhDs on certain benchmarks of knowledge and reasoning. They may…
So questioning the utility of LLMs for knowledge work is now akin to a conspiracy theory?
Re: Two things LLM coding agents are still bad at
#314Earlier quoted context omitted.
I think we need better code review tools in the age of LLMs - not just sticking another LLM to do a code review on top of the PR Needs to clearly handle the large diffs they produce - anyone have any ideas
I was about to write my own tool for this but then I discovered: git diff --color-moved=dimmed-zebra That shows a lot of code that was properly moved/copied in gray (even if it's an insertion). So gray stuff exactly matches something that was there before. Can also be enabled by default in the git config.
Perfect. This is why I visit this website
Re: Two things LLM coding agents are still bad at
#315Earlier quoted context omitted.
> I don't think they were ever really sold as that, and we have better tools for that. We have OpenAI calling gpt5 as having PhD level of intelligence and others like Anthropoc saying it will write all our code within months. Some are claiming it’s already writing 70%. I say they are being sold as a magical do everything tool.
They are lying, because their salary depends on them lying about it. Why does it even matter what they're saying? Why don't we listen to scientists, researchers, practicioners and the real users of the technology and stop repeating what the CEOs are saying? The things they're saying are technically correct, the best kind of correct. The models beat human PhDs on certain benchmarks of knowledge and reasoning. They may…
> When you see on tv that 9 out of 10 dentists recommend a toothpaste what do you do? Do you claim that brushing your teeth is a useless hype that's being pushed by big-tooth because they're exaggerating or misrepresenting what that means?
Only after schizophrenic dentists go around telling people that brushing their teeth is going to lead to a post-scarcity Star Trek world.
Re: Two things LLM coding agents are still bad at
#316On a more important level, I found that they still do really badly at even a minorly complex task without extreme babysitting. I wanted it to refactor a parser in a small project (2.5K lines total) because it'd gotten a bit too interconnected. It made a plan, which looked reasonable, so I told it to do this in stages, with checkpoints. It said it'd done so. I asked it "so is the old architecture also removed?" "No, i…
> I wanted it to refactor a parser in a small project This expression tree parser (typescript to sql query builder - https://tinqerjs.org/ ) has zero lines of hand-written code. It was made with Codex + Claude over two weeks (part-time on the side). Having worked on ORMs previously, it would have taken me 4x-10x the time to get to the same state (which also has 100s of tests, with some repetitions). That's a massive…
Development tools and libraries seem like they may be one of the absolute easiest use cases to get LLMs to work with since they generally have far less ambiguous requirements than other software and the LLMs generally have an enormous amount of data in their training set to help them understand the domain.
Re: Two things LLM coding agents are still bad at
#317Just the other day I hit something that I hadn't realized could happen. It was not code related in my case, but could happen with code or code-related things (and did to a coworker). In a discussion here on HN about why a regulation passed 15 years ago was not as general as it could have been, I speculated [1] that it could be that the technology at the time was not up to handling the general case and so they regulat…
They will just make up links. You need to make sure they're actually researching pages. That's what the deep research mode does. That being said, their interpretation of the information in the links is still influenced by their training.
Re: Two things LLM coding agents are still bad at
#318Earlier quoted context omitted.
They are lying, because their salary depends on them lying about it. Why does it even matter what they're saying? Why don't we listen to scientists, researchers, practicioners and the real users of the technology and stop repeating what the CEOs are saying? The things they're saying are technically correct, the best kind of correct. The models beat human PhDs on certain benchmarks of knowledge and reasoning. They may…
> When you see on tv that 9 out of 10 dentists recommend a toothpaste what do you do? Do you claim that brushing your teeth is a useless hype that's being pushed by big-tooth because they're exaggerating or misrepresenting what that means? Only after schizophrenic dentists go around telling people that brushing their teeth is going to lead to a post-scarcity Star Trek world.
It's a new technology which lends itself well to outrageous claims and marketing, but the analogy stands. The CEOs don't get to define the narrative or stand as strawman targets for anti-AI folks to dunk on, sorry. Elon has been repeating "self driving next year" for a decade+ at this point, that doesn't make what Waymo did unimpressive. This level of cynicism is unwarranted is what I'm saying.
Re: Two things LLM coding agents are still bad at
#319Earlier quoted context omitted.
I think we need better code review tools in the age of LLMs - not just sticking another LLM to do a code review on top of the PR Needs to clearly handle the large diffs they produce - anyone have any ideas
I was about to write my own tool for this but then I discovered: git diff --color-moved=dimmed-zebra That shows a lot of code that was properly moved/copied in gray (even if it's an insertion). So gray stuff exactly matches something that was there before. Can also be enabled by default in the git config.
Thanks:)
Re: Two things LLM coding agents are still bad at
#320I feel like the copy and paste thing is overdue a solution. I find this one particularly frustrating when working directly with ChatGPT and Claude via their chat interfaces. I frequently find myself watching them retype 100+ lines of code that I pasted in just to make a one line change. I expect there are reasons this is difficult, but difficult problems usually end up solved in the end.
> I frequently find myself watching them retype 100+ lines of code that I pasted in just to make a one line change.
In such cases, I specifically instruct LLMs to "only show the lines you would change" and they are very good at doing just that and eliding the rest. However, I usually do this after going through a couple of rounds of what you just described :-)
I partly do this to save time and partly to avoid using up more tokens. But I wonder if it is actually saving tokens given that hidden "thinking tokens" are a thing these days. That is, even if they do elide the unchanged code, I'm pretty sure they are "reasoning" about it before identifying only the relevant tokens to spit out.
As such, that does seem different from copy-and-paste tool use, which I believe is also solved. LLMs can already identify when code changes can be made programmatically... and then do so! I have actually seen ChatGPT write Python code to refactor other Python code: https://www.linkedin.com/posts/kunalkandekar_metaprogramming...
I had to fix a minor bug in its Python script to make it work, but it worked and was a bit of a moment for me. I still wonder if this is part of its system prompt or an emergent tool-use behavior. In either case, copy-and-paste seems like a much simpler problem that could be solved with specific prompting.