Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two things LLM coding agents are still bad at

kix.dev

Post reply on HN
Print view View original

281–290 of 382 posts

Re: Two things LLM coding agents are still bad at

#281

Post by ewoodrich »

tjansen wrote: post #18

Agreed with the points in that article, but IMHO the no 1 issue is that agents only see a fraction of the code repository. They don't know whether there is a helper function they could use, so they re-implement it. When contributing to UIs, they can't check the whole UI to identify common design patterns, so they re-invent it. The most important task for the human using the agent is to provide the right context. "Loo…

> When the agents needs to run commands like 'npm test' in a sub-directory, they almost never get it right the first time)

I was running into this constantly on one project with a repo split between a Vite/React front end and .NET backend (with well documented structure). It would sometimes go into panic mode after some npm command didn’t work repeatedly and do all sorts of pointless troubleshooting over and over, sometimes veering into destructive attempts to rebuild whatever it thought was missing/broken.

I kept trying to rewrite the section in CLAUDE.md to effectively instruct it to always first check the current directory to verify it was in the correct $CLIENT or $SERVER directory. But it would still sometimes forget randomly which was aggravating.

I ended up creating some aliases like “run-dev server restart” “run-dev client npm install” for common operations on both server/client that worked in any directory. Then added the base dotnet/npm/etc commands to the deny list which forced its thinking to go “Hmm it looks like I’m not allowed to run npm, so I’ll review the project instructions. I see, I can use the ‘run-dev’ helper to do $NPM_COMMAND…”

It’s been working pretty reliably now but definitely wasted a lot of time with a lot of aggravation getting to that solution.

Re: Two things LLM coding agents are still bad at

#282

Post by AIorNot »

scottbez1 wrote: post #247
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

The last point I think is most important: "very subtle and silently introduced mistakes" -- LLMs may be able to complete many tasks as well (or better) than humans, but that doesn't mean they complete them the same way, and that's critically important when considering failure modes. In particular, code review is one layer of the conventional swiss cheese model of preventing bugs, but code review becomes much less eff…

I think we need better code review tools in the age of LLMs - not just sticking another LLM to do a code review on top of the PR

Needs to clearly handle the large diffs they produce - anyone have any ideas

Re: Two things LLM coding agents are still bad at

#283

Post by atentaten »

cheema33 wrote: post #71

From the article: > I contest the idea that LLMs are replacing human devs... AI is not able to replace good devs. I am assuming that nobody sane is claiming such a thing today. But, it can probably replace bad and mediocre devs. Even today. In my org we had 3 devs who went through a 6-month code boot camp and got hired a few years ago when it was very difficult to find good devs. They struggled. I would give them eas…

>But we have a lot more food and a larger variety available. Technology made that possible.

Sure, but the food is less nutritious and more toxic.

Re: Two things LLM coding agents are still bad at

#284

Post by ewoodrich »

alzoid wrote: post #179
yodsanklai wrote: post #153

Earlier quoted context omitted.

5 minutes ago, I asked Claude to add some debug statements in my code. It also silently changed a regex in the code. It was easily caught with the diff but can be harder to spot in larger changes.

I asked Claude to add a debug endpoint to my hardware device that just gave memory information. It wrote 2600 lines of C that gave information about every single aspect of the system. On the one hand kind of cool. It looked at the MQTT code and the update code, the platform (esp) and generated all kinds of code. It recommended platform settings that could enable more detailed information that checked out when I looke…

Hah I also happened to use Claude recently to write basic MQTT code to expose some data on a couple Orange Pis I wanted to view in Home Assistant. And it one-shot this super cool mini Python MQTT client I could drop wherever I needed it which was amazing having never worked with MQTT in Python before.

I made some charts/dashboards in HA and was watching it in the background for a few minutes and then realized that none of the data was changing, at all.

So I went and looked at the code and the entire block that was supposed to pull the data from the device was just a stub generating test data based on my exact mock up of what I wanted the data it generated to look like.

Claude was like, “That’s exactly right, it’s a stub so you can replace it with the real data easily, let me know if you need help with that!” And to its credit, it did fix it to use actual data but I re-read my original prompt was somewhat baffling to think it could have been interpreted as wanting fake data given I explicitly asked it to use real data from the device.

Re: Two things LLM coding agents are still bad at

#285

Post by hbn »

I recently found a fun CLI application and was playing with it when I found out it didn't have proper handling for when you passed it invalid files, and spat out a cryptic error from an internal library which isn't a great UX.

I decided to pull the source code and fix this myself. It's written in Swift which I've used very little before, but this wasn't gonna be too complex of a change. So I got some LLMs to walk me through the process of building CLI apps in Xcode, code changes that need to be made, and where the build artifact is put in my filesystem so I could try it out.

I was able to get it to compile, navigate to my compiled binary, and run it, only to find my changes didn't seem to work. I tried everything, asking different LLMs to see if they can fix the code, spit out the binary's metadata to confirm the creation date is being updated when I compile, etc. Generally when I'd paste the code to an LLM and ask why it doesn't work it would assert the old code was indeed flawed, and my change needed to be done in X manner instead. Even just putting a print statement, I couldn't get those to run and the LLM would explain that it's because of some complex multithreading runtime gotcha that it isn't getting to the print statements.

After way too much time trouble-shooting, skipping dinner and staying up 90 minutes past when I'm usually in bed, I finally solved it - when I was trying to run my build from the build output directory, I forgot to put the ./ before the binary name, so I was running my global install from the developer and not the binary in the directory I was in.

Sure, rookie mistake, but the thing that drives me crazy with an LLM is if you give it some code and ask why it doesn't work, they seem to NEVER suggest it should actually be working, and instead will always say the old code is bad and here's the perfect fixed version of the code. And it'll even make up stuff about why the old code should indeed not work when it should, like when I was putting the print statements.

Re: Two things LLM coding agents are still bad at

#286

Post by nedrylandJP »

mcpeepants wrote: post #186
flowingfocus wrote: post #184

Earlier quoted context omitted.

A diff makes these kind of errors much easier to catch. Or maybe someone from XEROX has a better idea how to catch subtly altered numbers?

I verify all dates manually by memorizing their offset from the date of the signing of the Magna Carta

HN is no place for chicanery.

Re: Two things LLM coding agents are still bad at

#287

Post by qingcharles »

yodsanklai wrote: post #153
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

5 minutes ago, I asked Claude to add some debug statements in my code. It also silently changed a regex in the code. It was easily caught with the diff but can be harder to spot in larger changes.

I asked it to change some networking code, which it did perfectly, but I noticed some diffs in another file and found it had just randomly expanded some completely unrelated abbreviations in strings which are specifically shortened because of the character limit of the output window.

Re: Two things LLM coding agents are still bad at

#288

Post by [deleted] »

Romario77 wrote: post #222
dkarl wrote: post #208

Earlier quoted context omitted.

I've had similar experience both in coding and in non-coding research questions. An LLM will do the first N right and fake its work on the rest. It even happens when asking an LLM to reformat a document, or asking it to do extra research to validate information. For example, before a recent trip to another city, I asked Gemini to prepare a list of brewery taprooms with certain information, and I discovered it had inc…

LLMs are not good at "cycles" - when you have to go over a list and do the same action on each item. It's like it has ADHD and forgets or gets distracted in the middle. And the reason for that is that LLMs don't have memory and process the tokens, so as they keep going over the list the context becomes bigger with more irrelevant information and they can lose the reason they are doing what they are doing.

[deleted]

Re: Two things LLM coding agents are still bad at

#290

Post by qingcharles »

freetonik wrote: post #34

I see a pattern in these discussions all the time: some people say how very, very good LLMs are, and others say how LLMs fail miserably; almost always the first group presents examples of simple CRUD apps, frontend "represent data using some JS-framework" kind of tasks, while the second group presents examples of non-trivial refactoring, stuff like parsers (in this thread), algorithms that can't be found in leetcode,…

I use LLMs to vibe-code entire tools that I need for my work. They're really banal boring apps that are relatively simple, but they still would have wasted a day or two each to write and debug. Even stuff as simple as laying out the whole UI in a nice pattern. Most of these are now practically one-shots from the latest Claude and GPT. I leave them churning, get coffee, come back and test the finished product.
Post reply on HN