Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two things LLM coding agents are still bad at

kix.dev

Post reply on HN
Print view View original

181–190 of 382 posts

Re: Two things LLM coding agents are still bad at

#181

Post by an0malous »

quietbritishjim wrote: post #42
freetonik wrote: post #34

I see a pattern in these discussions all the time: some people say how very, very good LLMs are, and others say how LLMs fail miserably; almost always the first group presents examples of simple CRUD apps, frontend "represent data using some JS-framework" kind of tasks, while the second group presents examples of non-trivial refactoring, stuff like parsers (in this thread), algorithms that can't be found in leetcode,…

Yesterday, I got Claude Code to make a script that tried out different point clustering algorithms and visualise them. It made the odd mistake, which it then corrected with help, but broadly speaking it was amazing. It would've taken me at least a week to write by hsnd, maybe longer. It was writing the algorithms itself, definitely not just simple CRUD stuff.

Let’s see the diff

Re: Two things LLM coding agents are still bad at

#182

Post by pengfeituan »

The first issue is related to the inner behavior of LLMs. Human can ignore some detailed contents of code and copy and paste, but LLM convert them into hidden states. It is a process of compression. And the output is a process of decompression. And something maybe lost. So it is hard for LLM to copy and paste. The agent developer should customize the edit rules to do this.

The second issue is that, LLM does not learn much high level context relationship of knowledge. This can be improved by introducing more patterns in the training data. And current LLM training is doing much on this. I don't think it is a problem in next years.

Re: Two things LLM coding agents are still bad at

#183

Post by jihadjihad »

yodsanklai wrote: post #153
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

5 minutes ago, I asked Claude to add some debug statements in my code. It also silently changed a regex in the code. It was easily caught with the diff but can be harder to spot in larger changes.

I had a pretty long regex in a file that was old and crusty, and when I had Claude add a couple helpers to the file, it changed the formatting of the regex to be a little easier on the eyes in terms of readability.

But I just couldn't trust it. The diff would have been no help since it went from one long gnarly line to 5 tight lines. I kept the crusty version since at least I am certain it works.

Re: Two things LLM coding agents are still bad at

#184

Post by flowingfocus »

weinzierl wrote: post #136
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

Not code, but I once pasted an event announcement and asked for just spelling and grammar check. LLM suggested a new version with minor tweak which I copy pasted back. Just before sending I noticed that it had moved the event date by one day. Luckily I caught it but it taught me that you never should blindly trust LLM output even with super simple tasks, no relevant context size, clear and simple one sentence prompt.…

A diff makes these kind of errors much easier to catch.

Or maybe someone from XEROX has a better idea how to catch subtly altered numbers?

Re: Two things LLM coding agents are still bad at

#185

Post by enraged_camel »

First point is very annoying, yes, and it's why for large refactors I have the AI write step-by-step instructions and then do it myself. It's faster, cheaper and less error-prone.

The second point is easily handled with proper instructions. My AI agents always ask questions about points I haven't clarified, or when they come across a fork in the road. Frequently I'll say "do X" and it'll proceed, then halfway it will stop and say "I did some of this, but before I do the rest, you need to decide what to do about such and such". So it's a complete non-problem for me.

Re: Two things LLM coding agents are still bad at

#186

Post by mcpeepants »

flowingfocus wrote: post #184
weinzierl wrote: post #136

Earlier quoted context omitted.

Not code, but I once pasted an event announcement and asked for just spelling and grammar check. LLM suggested a new version with minor tweak which I copy pasted back. Just before sending I noticed that it had moved the event date by one day. Luckily I caught it but it taught me that you never should blindly trust LLM output even with super simple tasks, no relevant context size, clear and simple one sentence prompt.…

A diff makes these kind of errors much easier to catch. Or maybe someone from XEROX has a better idea how to catch subtly altered numbers?

I verify all dates manually by memorizing their offset from the date of the signing of the Magna Carta

Re: Two things LLM coding agents are still bad at

#187

Post by ravila4 »

cpfohl wrote: post #172
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

My custom prompt instructs GPT to output changes to code as a diff/git-patch. I don’t use agents because it makes it hard to see what’s happening and I don’t trust them yet.

I’ve tried this approach when working in chat interfaces (as opposed to IDEs), but I often find it tricky to review diffs without the full context of the codebase.

That said, your comment made me realize I could be using “git apply”more effectively to review LLM-generated changes directly in my repo. It’s actually a neat workflow!

Re: Two things LLM coding agents are still bad at

#188

Post by Gazoche »

NitpickLawyer wrote: post #52
freetonik wrote: post #34

I see a pattern in these discussions all the time: some people say how very, very good LLMs are, and others say how LLMs fail miserably; almost always the first group presents examples of simple CRUD apps, frontend "represent data using some JS-framework" kind of tasks, while the second group presents examples of non-trivial refactoring, stuff like parsers (in this thread), algorithms that can't be found in leetcode,…

> almost always the first group presents examples of simple CRUD apps How about a full programming language written by cc "in a loop" in ~3 months? With a compiler and stuff? https://cursed-lang.org/ It might be a meme project, but it's still impressive as hell we're here. I learned about this from a yt content creator that took that repo, asked cc to "make it so that variables can be emojis", and cc did that 5$ late…

> written by cc "in a loop" in ~3 months?

What does that mean exactly? I assume the LLM was not left alone with its task for 3 months without human supervision.

Re: Two things LLM coding agents are still bad at

#189

Post by IanCal »

hansmayer wrote: post #175
IanCal wrote: post #162

Earlier quoted context omitted.

They easily check a bunch of those boxes. > why don´t we stop pretending that we as users are stupid and don´t know how to use them This is in response to someone who saw a bunch of URLs coming out of it and was surprised at a bunch of them being wrong. That's using the tool wrong . It's like being surprised that the top results in google/app store/play store aren't necessarily the best match for your query but actua…

The URLs being wrong in that specific case is one where they were using the "wrong tool". I can name you at least a dozen other cases from own experience, where too, they appear to be the wrong tool, for example for working with Terraform or for not exposing secrets by hardcoding them in the frontend. Et cetera. Many other people will have contributed thousands if not more similar but different cases. So what good ar…

> Many of us mastered the hard problem of navigating various abstraction layers of computer over the years, only to be told, we now effing dont know how to write a few sentences in English? Come on.

If you're trying to one shot stuff with a few sentences then yes you might be using these things wrong. I've seen people with PhDs fail to use google successfully to find things, were they idiots? If you're using them wrong you're using them wrong - I don't care how smart you are in other areas. If you can't hand off work knowing someones capabilities then that's a thing you can't do - and that's ok. I've known unbelievably good engineers who couldn't form a solid plan to solve a business problem or collaboratively work to get something done to save their life. Those are different skills. But gpt5-codex and sonnet 4 / 4.5 can solidly write code, gpt-5-pro with web search can really dig into things, and if you can manage what they can do you can hand off work to them. If you've only ever worked with juniors with a feeling of "they slow everything down but maybe someday they'll be as useful as me" then you're less likely to succeed at this.

Let's do a quick overview of recent chats for me:

* Identifying and validating a race condition in some code

* Generating several approaches to a streaming issue, providing cost analyses of external services and complexity of 3 different approaches about how much they'd change the code

* Identifying an async bug two good engineers couldn't find in a codebase they knew well

* Finding performance issues that had gone unnoticed

* Digging through synapse documentation and github issues to find a specific performance related issue

* Finding the right MSC for a feature I wanted to use but didn't know existed - and then finding the github issue that explained how it was only half implemented and how to enable the experimental other part I needed

* Building a bunch of UI stuff for a short term contract I needed, saving me a bunch of hours and the client money

* Going through funding opportunities and matching them against a charity I want to help in my local area

* Building a search integration for my local library to handle my kids reading challenge

* Solving a series of VPN issues I didn't understand

* Writing a lot of astro related python for an art project to cover the loss of some NASA images I used to have access to.

> the folks pushing or better said

If you don't want to trust them, don't. Also don't believe the anti-hype merchants who want to smugly say these tools can't do a god damn thing. They're trying to get attention as well.

Re: Two things LLM coding agents are still bad at

#190

Post by goalieca »

IanCal wrote: post #167
grey-area wrote: post #59

Earlier quoted context omitted.

Or just not bother. It sounds pretty useless if it flunks on basic tasks like this. Perhaps you’ve been sold a lie?

They're moderately unreliable text copying machines if you need exact copying of long arbitrary strings. If that's what you want, don't use LLMs. I don't think they were ever really sold as that, and we have better tools for that. On the other hand, I've had them easily build useful code, answer questions and debug issues complex enough to escape good engineers for at least several hours. Depends what you want. They'…

> I don't think they were ever really sold as that, and we have better tools for that.

We have OpenAI calling gpt5 as having PhD level of intelligence and others like Anthropoc saying it will write all our code within months. Some are claiming it’s already writing 70%.

I say they are being sold as a magical do everything tool.

Post reply on HN