Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two things LLM coding agents are still bad at

kix.dev

Post reply on HN
Print view View original

151–160 of 382 posts

Re: Two things LLM coding agents are still bad at

#151

Post by smougel »

rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

Not related to code... But when I use a LLM to perform a kind of copy/paste, I try to number the lines and ask it to generate a start_index and stop_index to perform the slice operation. Much less hallucinations and very cheap in token generation.

Re: Two things LLM coding agents are still bad at

#152

Post by gessha »

notpachet wrote: post #143

> They keep trying to make it work until they hit a wall -- and then they just keep banging their head against it. This is because LLMs trend towards the centre of the human cognitive bell curve in most things, and a LOT of humans use this same problem solving approach.

The approach doesn’t matter as much. The halting problem does :)

Re: Two things LLM coding agents are still bad at

#153

Post by yodsanklai »

rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

5 minutes ago, I asked Claude to add some debug statements in my code. It also silently changed a regex in the code. It was easily caught with the diff but can be harder to spot in larger changes.

Re: Two things LLM coding agents are still bad at

#154

Post by mallowdram »

hotpotat wrote: post #131

Lol this person talks about easing into LLMs again two weeks after quitting cold turkey. The addiction is real. I laugh because I’m in the same situation, and see no way out other than to switch professions and/or take up programming as a hobby in which I purposefully subject myself to hard mode . I’m too productive with it in my profession to scale back and do things by hand — the cat is out of the bag and I’ve set…

cat out of the bag is disautomation. the speed in the timetable is an illusion if the supervision requires blast radius retention. this is more like an early video game assembly line than a structured skilled industry

Re: Two things LLM coding agents are still bad at

#155

Post by tuesdaynight »

cimi_ wrote: post #112
rossant wrote: post #79

Earlier quoted context omitted.

It was a fairly big refactoring basically converting a working static HTML landing page into a Hugo website, splitting the HTML into multiple Hugo templates. I admit I was quite in a hurry and had to take shortcuts. I didn't have time to write automated tests and had to rely on manual tests for this single webpage. The diff was fairly big. It just didn't occur to me that the URLs would go through the LLMs and could b…

Speaking of agents and tests, here's a fun one I had the other day: while refactoring a large code base I told the agent to do something precise to a specific module, refactor with the new change, then ensure the tests are passing. The test suite is slow and has many moving parts; the tests I asked it to run take ~5 minutes. The thing decided to kill the test run, then it made up another command it said was the 'test…

I think that it's something that model providers don't want to fix, because the amount of times that Claude Code just decided to delete tests that were not passing before I added a memory saying that it would need to ask for my permission to do that was staggering. It stopped happening after the memory, so I believe that it could be easily fixed by a system prompt.

Re: Two things LLM coding agents are still bad at

#156

Post by jollyllama »

ivape wrote: post #53
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

You’re just not using LLMs enough. You can never trust the LLM to generate a url, and this was known over two years ago. It takes one token hallucination to fuck up a url. It’s very good at a fuzzy great answer, not a precise one. You have to really use this thing all the time and pick up on stuff like that.

> You’re just not using LLMs enough.

> You can never trust the LLM to generate a url

This is very poorly worded. Using LLMs more wouldn't solve the problem. What you're really saying is that the GP is uninformed about LLMs.

This may seem like pedantry on my part but I'm sick of hearing "you're doing it wrong" when the real answer is "this tool can't do that." The former is categorically different than the latter.

Re: Two things LLM coding agents are still bad at

#157

Post by latexr »

mcny wrote: post #137

I sometimes give LLM random "easy" questions. My assessment is still that they all need the fine print "bla bla can be incorrect" You should either already know the answer or have a way to verify the answer. If neither, the matter must be inconsequential like just a child like curiosity. For example, I wonder how many moons Jupiter has... It could be 58, it could be 85 but either answer won't alter any of what I do t…

> For example, I wonder how many moons Jupiter has... It could be 58, it could be 85

For those curious, the answer is 97.

https://en.wikipedia.org/wiki/Moons_of_Jupiter

Re: Two things LLM coding agents are still bad at

#158

Post by iLoveOncall »

pprotas wrote: post #109
iLoveOncall wrote: post #84

Earlier quoted context omitted.

Hum yeah, it shows. Just the fact that the API looks completely different for Postgre and SQLite tells us everything we need to know about the quality of the project here.

I guess the interesting question is whether @jeswin could have created this project at all if AI tools were not involved. And if yes, would the quality even be better?

Actually the interesting question is whether this library not existing would have been a loss for humanity. I'll posit that it would not.

Re: Two things LLM coding agents are still bad at

#159

Post by rossant »

cimi_ wrote: post #112
rossant wrote: post #79

Earlier quoted context omitted.

It was a fairly big refactoring basically converting a working static HTML landing page into a Hugo website, splitting the HTML into multiple Hugo templates. I admit I was quite in a hurry and had to take shortcuts. I didn't have time to write automated tests and had to rely on manual tests for this single webpage. The diff was fairly big. It just didn't occur to me that the URLs would go through the LLMs and could b…

Speaking of agents and tests, here's a fun one I had the other day: while refactoring a large code base I told the agent to do something precise to a specific module, refactor with the new change, then ensure the tests are passing. The test suite is slow and has many moving parts; the tests I asked it to run take ~5 minutes. The thing decided to kill the test run, then it made up another command it said was the 'test…

So it took a shortcut as it was too lazy and it lied to your face about it. AGI is here for good.

Re: Two things LLM coding agents are still bad at

#160

Post by mexicocitinluez »

mcny wrote: post #137

I sometimes give LLM random "easy" questions. My assessment is still that they all need the fine print "bla bla can be incorrect" You should either already know the answer or have a way to verify the answer. If neither, the matter must be inconsequential like just a child like curiosity. For example, I wonder how many moons Jupiter has... It could be 58, it could be 85 but either answer won't alter any of what I do t…

> or have a way to verify the answer

Fortunately, as devs, this is our main loop. Write code, test, debug. And it's why people who fear AI-generated code making it's way into production and causing errors makes me laugh. Are you not testing your code? Or even debugging it? Like, what process are you using that prevents bugs happening? Guess what? It's the exact same process with AI-generated code.

Post reply on HN