Live data from Hacker News

Two things LLM coding agents are still bad at

kix.dev

361–370 of 382 posts

Re: Two things LLM coding agents are still bad at

#361
post #329

Earlier quoted context omitted.

A high school student would use copy/paste and the urls would be perfect duplicates..

LLMs aren’t high school students, they’re blobs of numbers which happen to speak English if you poke them right. Use the tool when it’s good at what it does.

And the people who are causing this confusion are the CEOs of the companies saying that the newest model is a PHD in your pocket.

Re: Two things LLM coding agents are still bad at

#362

Earlier quoted context omitted.

They are lying, because their salary depends on them lying about it. Why does it even matter what they're saying? Why don't we listen to scientists, researchers, practicioners and the real users of the technology and stop repeating what the CEOs are saying? The things they're saying are technically correct, the best kind of correct. The models beat human PhDs on certain benchmarks of knowledge and reasoning. They may…

> When you see on tv that 9 out of 10 dentists recommend a toothpaste what do you do? Do you claim that brushing your teeth is a useless hype that's being pushed by big-tooth because they're exaggerating or misrepresenting what that means? Only after schizophrenic dentists go around telling people that brushing their teeth is going to lead to a post-scarcity Star Trek world.

You shouldn’t - that’s the point of the comparison. If some insane dentists started saying this you should not stop brushing your teeth!

Re: Two things LLM coding agents are still bad at

#363

Earlier quoted context omitted.

Yeah but before AI overviews Google would have shown the first search result with a text snippet directly quoted from the page with the answer highlighted. Thats just as fast (or faster) than the AI overview

The snippet included in the search result does not include or highlight the relevant fact. I feel like you’re not willing to take simple actions to confirm your assertions.

When I searched, the top result was Wikipedia with the following excerpt: “At the 1974 federal election, Howard was elected as a member of parliament (MP) for the division of Bennelong. He was promoted to cabinet in 1977, and…”

To me this seemed like the relevant detail in the first excerpt.

But after more thought I realize you were probably expecting the date of his election to prime minister which is fair! That’s probably what searchers would be looking for.

Re: Two things LLM coding agents are still bad at

#364

Lol this person talks about easing into LLMs again two weeks after quitting cold turkey. The addiction is real. I laugh because I’m in the same situation, and see no way out other than to switch professions and/or take up programming as a hobby in which I purposefully subject myself to hard mode . I’m too productive with it in my profession to scale back and do things by hand — the cat is out of the bag and I’ve set…

In my defense, I wrote the blog post about quitting a good while after I've already quit cold turkey -- but you're spot on. :)

Especially when surrounded by people who swear LLMs can really be gamechanging on certain tasks, it's really hard to just keep doing things by hand (especially if you have the gut feeling that an LLM can probably do rote pretty well, based on past experience).

What kind of works for me now is what a colleague of mine calls "letting it write the leaf nodes in the code tree". So long as you take on the architecture, high level planning, schemas, and all the important bits that require thinking - chances are it can execute writing code successfully by following your idiot-proof blueprint. It's still a lot of toll and tedium, but perhaps still beats mechanical labor.

Re: Two things LLM coding agents are still bad at

#365
post #60

I'd argue LLM coding agents are still bad at many more things. But to comment on the two problems raised in the post: > LLMs don’t copy-paste (or cut and paste) code. The article is confusing the architectural layers of AI coding agents. It's easy to add "cut/copy/paste" tools to the AI system if that shows improvement. This has nothing to do with LLM, it's in the layer on top. > Good human developers always pause to…

> The article is confusing the architectural layers of AI coding agents. It's easy to add "cut/copy/paste" tools to the AI system if that shows improvement. This has nothing to do with LLM, it's in the layer on top.

I think we can't trivialize adding good cut/copy/paste tools though. It's not like we can just slap those tools on the topmost layer (ex, on Claude Code, Codex, or Roo) and it'll just work.

I think that a lot of reinforcement learning that LLM providers do on their coding models barely (if at all) steer towards that kind of tool use, so even if we implemented those tools on top of coding LLMs they probably would just splash and do nothing.

Adding cut/copy/paste probably requires a ton of very specific (and/or specialized) fine tuning with not a ton of data to train on -- think recordings of how humans use IDEs, keystrokes, commands issued, etc etc.

I'm guessing Cursor's Autocomplete model is the closest thing that can do something like this if they chose to, based on how they're training it.

Re: Two things LLM coding agents are still bad at

#366
post #272

Just the other day I hit something that I hadn't realized could happen. It was not code related in my case, but could happen with code or code-related things (and did to a coworker). In a discussion here on HN about why a regulation passed 15 years ago was not as general as it could have been, I speculated [1] that it could be that the technology at the time was not up to handling the general case and so they regulat…

It gets more obvious once you start researching stuff that is quite niche, like how to connect a forgotten old USB device to a modern computer and the only person posting about it was a Russian guy on an almost abandoned forum.

Re: Two things LLM coding agents are still bad at

#367
post #327

Earlier quoted context omitted.

Or just not bother. It sounds pretty useless if it flunks on basic tasks like this. Perhaps you’ve been sold a lie?

Read about the jagged frontier. IanCal is right: this is a perfect example of using the tool wrong; you’ve focused on a very narrow use case which is surprisingly hard for the matmuls to not mess up and extrapolate, but extrapolation is incorrect here because the capability frontier is fractal and not continuous.

It’s not surprisingly hard at all, when you consider they have no understanding of the tasks they do nor of the subject material. It’s just a good example of the types of tasks (anything requiring reliability or correct results) that they are fundamentally unsuited to.

Sadly it seems the best use-case for LLMs at this point is bamboozling humans.

Re: Two things LLM coding agents are still bad at

#368
post #327

Earlier quoted context omitted.

Read about the jagged frontier. IanCal is right: this is a perfect example of using the tool wrong; you’ve focused on a very narrow use case which is surprisingly hard for the matmuls to not mess up and extrapolate, but extrapolation is incorrect here because the capability frontier is fractal and not continuous.

It’s not surprisingly hard at all, when you consider they have no understanding of the tasks they do nor of the subject material. It’s just a good example of the types of tasks (anything requiring reliability or correct results) that they are fundamentally unsuited to. Sadly it seems the best use-case for LLMs at this point is bamboozling humans.

When you take a step back it's surprising that these tools can be actually useful at all in nontrivial tasks, but being surprised doesn't matter in the grand scheme of things. Bamboozling rarely enough for harnesses to keep them in line and ability to inference-time self-correct when bamboozling is detected either by the model itself or by the harness is very useful at least in my work. It's a question of using the tool correctly and understanding its limitations, which is hard if you aren't willing to explore the boundaries and commit to doing it every month basically.

Re: Two things LLM coding agents are still bad at

#369

Earlier quoted context omitted.

I regularly check in on using LLMs. But a key criteria for me is that an LLM needs to objectively make me more efficient, not subjectively . Often I find myself cursing at the LLM for not understanding what I mean - which is expensive in lost time / cost of tokens. It is easy to say: Then just don't use LLMs. But in reality, it is not too easy to break out of these loops of explaining, and it is extremely hard to ass…

> Often I find myself cursing at the LLM for not understanding what I mean... Me too. But in all these cases, sooner or later, I realized I made a mistake not giving enough context and not building up the discussion carefully enough. And I was just rushing to the solution. In the agile world, one could say I gave the LLM not a well-defined story, but a one-liner. Who is to blame here? I still remember training a juni…

> ... Who is to blame here?

That is not really relevant, is it? The LLM is not a human.

The question is whether it is still af efficient to use LLMs after spending huge amounts of time giving the context - or if it is just as efficient to write the code yourself.

> I still remember training a junior hire who started off with

Working with LLMs is not training junior developers - treating it as such is yet another resource sink.

Post reply on HN