Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two things LLM coding agents are still bad at

kix.dev

Post reply on HN
Print view View original

291–300 of 382 posts

Re: Two things LLM coding agents are still bad at

#291

Post by strangescript »

justonceokay wrote: post #223
strangescript wrote: post #221

You don't want your agents to ask questions. You are thinking too short term. Its not ideal now, but agents that have to ask frequent questions are useless when it comes the vision of totally autonomous coding. Humans ask questions of groups to fix our own personal short comings. It make no sense to try and master an internal system I rarely use, I should instead ask someone that maintains it. AI will not have this p…

If you look at a piece of architecture, you might be able to infer the intentions of the architect. However, there are many interpretations possible. So if you were to add an addendum to the building it makes sense that you might want to ask about the intentions. I do not believe that AI will magically overcome the Chesterton Fence problem in a 100% autonomous way.

AI won't, but humans will to un-encumber AI

Re: Two things LLM coding agents are still bad at

#292

Post by slyzmud »

freetonik wrote: post #34

I see a pattern in these discussions all the time: some people say how very, very good LLMs are, and others say how LLMs fail miserably; almost always the first group presents examples of simple CRUD apps, frontend "represent data using some JS-framework" kind of tasks, while the second group presents examples of non-trivial refactoring, stuff like parsers (in this thread), algorithms that can't be found in leetcode,…

The two groups are very different but I notice another pattern: you have people who like coding and understanding details of what their are doing, are curious, what to learn about the why and think about edge cases; and there's another group of people who just want to code something, make a test pass, show a nice UI and that's it, but don't think much about edge cases or maintainability. The only thing they think is "delivering value" to customers.

Usually those two groups correlate very well with liking LLMs: some people will ask Claude to create a UI with React and see the mess it generated (even if it mostly works) and the edge cases it left out and comment in forums that LLMs don't work. The other group of people will see the UI working and call it a day without even noticing the subtleties.

Re: Two things LLM coding agents are still bad at

#293

Post by steveklabnik »

AIorNot wrote: post #282
scottbez1 wrote: post #247

Earlier quoted context omitted.

The last point I think is most important: "very subtle and silently introduced mistakes" -- LLMs may be able to complete many tasks as well (or better) than humans, but that doesn't mean they complete them the same way, and that's critically important when considering failure modes. In particular, code review is one layer of the conventional swiss cheese model of preventing bugs, but code review becomes much less eff…

I think we need better code review tools in the age of LLMs - not just sticking another LLM to do a code review on top of the PR Needs to clearly handle the large diffs they produce - anyone have any ideas

I personally agree with you. I think that stacked diffs will be more important as a way of dealing with those larger diffs.

Re: Two things LLM coding agents are still bad at

#294

Post by bckmn »

tzs wrote: post #272

Just the other day I hit something that I hadn't realized could happen. It was not code related in my case, but could happen with code or code-related things (and did to a coworker). In a discussion here on HN about why a regulation passed 15 years ago was not as general as it could have been, I speculated [1] that it could be that the technology at the time was not up to handling the general case and so they regulat…

I think asking your questions in that form is akin to "sorting prompts" that I learned about from https://mikecaulfield.substack.com/p/is-the-llm-response-wro... and I have been using successfully when when writing code (e.g. [as a Claude code slash command](https://www.joshbeckman.org/notes/936274709)).

Essentially, you're asking the LLM to do research and categorize/evaluate that research instead of just giving you an answer. The "work" of accessing, summarizing, and valuing the research yields a more accurate result.

Re: Two things LLM coding agents are still bad at

#295

Post by steveklabnik »

polynomial wrote: post #235
Romario77 wrote: post #222

Earlier quoted context omitted.

LLMs are not good at "cycles" - when you have to go over a list and do the same action on each item. It's like it has ADHD and forgets or gets distracted in the middle. And the reason for that is that LLMs don't have memory and process the tokens, so as they keep going over the list the context becomes bigger with more irrelevant information and they can lose the reason they are doing what they are doing.

So much for Difference and Repetition.

Surprised and a bit delighted to see a Deleuze reference on HN...

Re: Two things LLM coding agents are still bad at

#296

Post by steveklabnik »

radarsat1 wrote: post #268
fwip wrote: post #239

Earlier quoted context omitted.

It would be nice if the tools we usually use for LLMs had a bit more programmability. In this example, It we could imagine being able to chunk up work by processing a few items, then reverting to a previous saved LLM checkpoint of state, and repeating until the list is complete. I imagine that the cost of saving & loading the current state must be prohibitively high for this to be a normal pattern, though.

Agreed. You basically want an LLM to have a tool that writes its own agent to accomplish a repetitive task. I think this is doable.

You can already sort of do this by asking it to write a script to do the refactor. Claude sometimes suggests this on its own to me even.

But obviously sometimes larger refactors aren't easy to implement in bash.

Re: Two things LLM coding agents are still bad at

#297

Post by colonCapitalDee »

stuartjohnson12 wrote: post #230
troupo wrote: post #214

Earlier quoted context omitted.

> About half of the code generated fake data rather than actually do anything with the system. All the time // fake data. in production this would be real data ... proceeds to write sometimes hundreds of lines of code to provide fake data

"hey claude, please remove the fake data and use the real data" "sure thing, I'll add logic to check if the real data exists and only use the fake data as a fallback in case the real data doesn't exist"

Claude (possible all LLMs, but I mostly use Claude) LOVES this pattern for some reason. "If fails/does not exist I'll just silently return a placeholder, that way things break silently and you'll tear your hair out debugging it later!" Thanks Claude

Re: Two things LLM coding agents are still bad at

#298

Post by qnleigh »

rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

Interesting, I've seen similar looking behavior in other forms of data extraction. I took a picture of a bookshelf and asked it to list the books. It did well in the beginning but by the middle, it had started making up similar books that were not actually there.

Re: Two things LLM coding agents are still bad at

#299

Post by svieira »

jeswin wrote: post #68
AllegedAlec wrote: post #13

On a more important level, I found that they still do really badly at even a minorly complex task without extreme babysitting. I wanted it to refactor a parser in a small project (2.5K lines total) because it'd gotten a bit too interconnected. It made a plan, which looked reasonable, so I told it to do this in stages, with checkpoints. It said it'd done so. I asked it "so is the old architecture also removed?" "No, i…

> I wanted it to refactor a parser in a small project This expression tree parser (typescript to sql query builder - https://tinqerjs.org/ ) has zero lines of hand-written code. It was made with Codex + Claude over two weeks (part-time on the side). Having worked on ORMs previously, it would have taken me 4x-10x the time to get to the same state (which also has 100s of tests, with some repetitions). That's a massive…

Quite impressive, thank you for sharing!

Question - this loads a 2 MB JS parser written in Rust to turn `x => x.foo` into `{ op: 'project', field: 'foo', target: 'x' }`. But you don't actually allow any complex expressions (and you certainly don't seem to recursively parse references or allow return uplift, e. g. I can't extract out `isOver18` or `isOver(age: int)(Row: IQueryable): IQueryable`). Why did you choose the AST route instead of doing the same thing with a handful of regular expressions?

Re: Two things LLM coding agents are still bad at

#300

Post by colonCapitalDee »

weakfish wrote: post #259
username223 wrote: post #232

Earlier quoted context omitted.

> Any need we had for a junior dev has completely disappeared. Where do your senior devs come from?

That's the question that has been stuck in my head as I read all these stories about junior dev jobs disappearing. I'm firmly mid-level, having started my career just before LLM coding took off. Sometimes it feels like I got on the last chopper out of Saigon.

Yep, I graduated and got my first job in 2022 when the market was hot and ChatGPT was a fun novelty. Very lucky
Post reply on HN