Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two things LLM coding agents are still bad at

kix.dev

Post reply on HN
Print view View original

81–90 of 382 posts

Re: Two things LLM coding agents are still bad at

#81

Post by saghm »

stellalo wrote: post #64
cat-whisperer wrote: post #45

The copy-paste thing is interesting because it hints at a deeper issue: LLMs don't have a concept of "identity" for code blocks—they just regenerate from learned patterns. I've noticed similar vibes when agents refactor—they'll confidently rewrite a chunk and introduce subtle bugs (formatting, whitespace, comments) that copy-paste would've preserved. The "no questions" problem feels more solvable with better promptin…

I feel like it’s the opposite: the copy-paste issue is solvable, you just need to equip the model with the right tools and make sure they are trained on tasks where that’s unambiguously the right thing to do (for example, cases were copying code “by hand” would be extremely error prone -> leads to lower reward on average). On the other hand, teaching the model to be unsure and ask questions, requires the training loo…

> On the other hand, teaching the model to be unsure and ask questions, requires the training loop to break and bring a human input in, which appears more difficult to scale.

The ironic thing to me is that the one thing they never seem to be willing to skip asking about is whether they should proceed with some fix that I just helped them identify. They seem extremely reluctant to actually ask about things they don't know about, but extremely eager to ask about whether they should do the things they already have decided they think are right!

Re: Two things LLM coding agents are still bad at

#82

Post by janmarsal »

My biggest issue with LLMs right now is that they're such spineless yes men. Even when you ask their opinion on if something is doable or should it be done in the first place, more often than not they just go "Absolutely!" and shit out a broken answer or an anti-pattern just to please you. Not always, but way too often. You need to frame your questions way too carefully to prevent this.

Maybe some of those character.ai models are sassy enough to have stronger opinions on code?

Re: Two things LLM coding agents are still bad at

#83

Post by coldtea »

rglynn wrote: post #26
AllegedAlec wrote: post #19

Earlier quoted context omitted.

Claude code. Whichever model it started up automatically last weekend, I didn't explicitly check.

This feels like a classic Sonnet issue. From my experience, Opus or GPT-5-high are less likely to do the "narrow instruction following without making sensible wider decisions based on context" than Sonnet.

This is "just use another Linux distro" all over again

Re: Two things LLM coding agents are still bad at

#84

Post by iLoveOncall »

jeswin wrote: post #68
AllegedAlec wrote: post #13

On a more important level, I found that they still do really badly at even a minorly complex task without extreme babysitting. I wanted it to refactor a parser in a small project (2.5K lines total) because it'd gotten a bit too interconnected. It made a plan, which looked reasonable, so I told it to do this in stages, with checkpoints. It said it'd done so. I asked it "so is the old architecture also removed?" "No, i…

> I wanted it to refactor a parser in a small project This expression tree parser (typescript to sql query builder - https://tinqerjs.org/ ) has zero lines of hand-written code. It was made with Codex + Claude over two weeks (part-time on the side). Having worked on ORMs previously, it would have taken me 4x-10x the time to get to the same state (which also has 100s of tests, with some repetitions). That's a massive…

Hum yeah, it shows. Just the fact that the API looks completely different for Postgre and SQLite tells us everything we need to know about the quality of the project here.

Re: Two things LLM coding agents are still bad at

#85

Post by jansan »

AllegedAlec wrote: post #13

On a more important level, I found that they still do really badly at even a minorly complex task without extreme babysitting. I wanted it to refactor a parser in a small project (2.5K lines total) because it'd gotten a bit too interconnected. It made a plan, which looked reasonable, so I told it to do this in stages, with checkpoints. It said it'd done so. I asked it "so is the old architecture also removed?" "No, i…

I was hoping that LLMs being able to access strict tools, like Gemini using Python libraries, would finally give reliable results.

So today I asked Gemini to simplify a mathematical expression with sympy. It did and explained to me how some part of the expression could be simplified wonderfully as a product of two factors.

But it was all a lie. Even though I explicitly asked it to use sympy in order to avoid such hallucinations and get results that are actually correct, it used its own flawed reasoning on top and again gave me a completely wrong result.

You still cannot trust LLMs. And that is a problem.

Re: Two things LLM coding agents are still bad at

#86

Post by freetonik »

NitpickLawyer wrote: post #52
freetonik wrote: post #34

I see a pattern in these discussions all the time: some people say how very, very good LLMs are, and others say how LLMs fail miserably; almost always the first group presents examples of simple CRUD apps, frontend "represent data using some JS-framework" kind of tasks, while the second group presents examples of non-trivial refactoring, stuff like parsers (in this thread), algorithms that can't be found in leetcode,…

> almost always the first group presents examples of simple CRUD apps How about a full programming language written by cc "in a loop" in ~3 months? With a compiler and stuff? https://cursed-lang.org/ It might be a meme project, but it's still impressive as hell we're here. I learned about this from a yt content creator that took that repo, asked cc to "make it so that variables can be emojis", and cc did that 5$ late…

Ok, not trivial for sure, but not novel? IIUC, the language does not have really new concepts, apart from the keywords (which is trivial).

Impressive nonetheless.

Re: Two things LLM coding agents are still bad at

#87

Post by [deleted] »

AllegedAlec wrote: post #13

On a more important level, I found that they still do really badly at even a minorly complex task without extreme babysitting. I wanted it to refactor a parser in a small project (2.5K lines total) because it'd gotten a bit too interconnected. It made a plan, which looked reasonable, so I told it to do this in stages, with checkpoints. It said it'd done so. I asked it "so is the old architecture also removed?" "No, i…

[deleted]

Re: Two things LLM coding agents are still bad at

#88

Post by seunosewa »

sidgtm wrote: post #36

As a UX designer I see they lack the ability of being opinionated about a design piece and go with the standard mental model. I got fed up with this and made a simple java script code to run a simple canvas on the localhost to pass on more subjective feedback using highlights and notes feature. I tried using playwright first but a. its token heavy b. it's still for finding what's working or breaking instead of thinki…

Whatdo the notes look like?

Re: Two things LLM coding agents are still bad at

#89

Post by hansmayer »

ivape wrote: post #53
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

You’re just not using LLMs enough. You can never trust the LLM to generate a url, and this was known over two years ago. It takes one token hallucination to fuck up a url. It’s very good at a fuzzy great answer, not a precise one. You have to really use this thing all the time and pick up on stuff like that.

Yeah so, the reason people use various tools and machines in the first place is to simplify the work or everydays tasks by : 1) Making the tasks execute faster 2) Getting more reliable outputs then doing this by yourself 3) Making it repeatable . The LLMs obviously dont check any of these boxes so why don´t we stop pretending that we as users are stupid and don´t know how to use them and start taking them for what they are - cute little mirages, perhaps applicable as toys of some sort, but not something we should use for serious engineering work really?

Re: Two things LLM coding agents are still bad at

#90

Post by ivape »

hansmayer wrote: post #89
ivape wrote: post #53

Earlier quoted context omitted.

You’re just not using LLMs enough. You can never trust the LLM to generate a url, and this was known over two years ago. It takes one token hallucination to fuck up a url. It’s very good at a fuzzy great answer, not a precise one. You have to really use this thing all the time and pick up on stuff like that.

Yeah so, the reason people use various tools and machines in the first place is to simplify the work or everydays tasks by : 1) Making the tasks execute faster 2) Getting more reliable outputs then doing this by yourself 3) Making it repeatable . The LLMs obviously dont check any of these boxes so why don´t we stop pretending that we as users are stupid and don´t know how to use them and start taking them for what th…

[flagged]
Post reply on HN