Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two things LLM coding agents are still bad at

kix.dev

Post reply on HN
Print view View original

301–310 of 382 posts

Re: Two things LLM coding agents are still bad at

#301

Post by sarchertech »

NitpickLawyer wrote: post #275
sarchertech wrote: post #270

Earlier quoted context omitted.

There’s no evidence that this ever happened other than this guy’s word. And since the claim that he ran an agent with no human intervention for 3 months is so far outside of any capabilities demonstrated by anyone else, I’m going to need to see some serious evidence before I believe it.

> There’s no evidence that this ever happened other than this guy’s word. There's a yt channel where the sessions were livestreamed. It's in their FAQ. I haven't felt the need to check them, but there are 10-12h sessions in there if you're that invested in proving that this is "so far outside of any capabilities"... A brief look at the commit history should show you that it's 99.9% guaranteed to be written by an LLM…

>When's the last time you used one of these SotA coding agents?

This morning :)

>"so far outside of any capabilities"

Anthropic was just bragging last week about being able to code without intervention for 30 hours before completely losing focus. They hailed it as a new bench mark. It completed a project that was 11k lines of code.

The max unsupervised run that GPT-5-Codex has been able to pull off is 7 hours.

That's what I mean by the current SOTA demonstrated capabilities.

https://x.com/rohanpaul_ai/status/1972754113491513481

And yet here you have a rando who is saying that he was able able to get an agent to run unsupervised for 100x longer than what the model companies themselves have been able to do and produce 10x the amount of code--months ago.

I'm 100% confident this is fake.

>There's a yt channel where the sessions were livestreamed.

There are a few videos that long, not 3 months worth of videos. Also I spot checked the videos and it the framerate is so low that it would be trivial to cut out the human intervention.

>guaranteed to be written by an LLM

I don't doubt that it was 99.9% written by an LLM, the question is whether he was able to run unsupervised for 3 months or whether he spent 3 months guiding an LLM to write it.

Re: Two things LLM coding agents are still bad at

#302

Post by gen220 »

How I describe this phenomenon:

If the code-change is something you would reasonably prefer to use a codemod to implement (i.e. dozens-to-hundreds of small changes fitting a semantic pattern), Claude Code not going to be able to make that change effectively.

However (!), CC is pretty good at writing the codemod.

Re: Two things LLM coding agents are still bad at

#303

Post by overgard »

I definitely feel the "bad at asking questions" part, a lot of times I'll walk away for a second while it's working, and then I come back and it's gone down some intricate path I really didn't want and if it had just asked a question at the right point it would have saved a lot of wasted work (plus I feel like having that "bad" work in the context window potentially leads to problems down the road). The problem is just that I'm pretty sure there isn't any way for an LLM to really be "uncertain" about a thing, it's basically always certain even when it's incredibly wrong.

To me, I think I'm fine just accepting them for what they're good at. I like them for generating small functions, or asking questions about a really weird error I'm seeing. I don't ever ask it to refactor things though, that seems like a recipe for disaster and a tool that understands the code structure is a lot better for moving things around then an LLM is.

Re: Two things LLM coding agents are still bad at

#304

Post by peterbonney »

"weird, overconfident interns" -> exactly the mental model I try to get people to use when thinking about LLM capabilities in ALL domains, not just coding.

A good intern is really valuable. An army of good interns is even more valuable. But interns are still interns, and you have to check their work. Carefully.

Re: Two things LLM coding agents are still bad at

#305

Post by NitpickLawyer »

sarchertech wrote: post #301
NitpickLawyer wrote: post #275

Earlier quoted context omitted.

> There’s no evidence that this ever happened other than this guy’s word. There's a yt channel where the sessions were livestreamed. It's in their FAQ. I haven't felt the need to check them, but there are 10-12h sessions in there if you're that invested in proving that this is "so far outside of any capabilities"... A brief look at the commit history should show you that it's 99.9% guaranteed to be written by an LLM…

>When's the last time you used one of these SotA coding agents? This morning :) >"so far outside of any capabilities" Anthropic was just bragging last week about being able to code without intervention for 30 hours before completely losing focus. They hailed it as a new bench mark. It completed a project that was 11k lines of code. The max unsupervised run that GPT-5-Codex has been able to pull off is 7 hours. That's…

I think you are confusing 2 things here. What the labs mean when they announce x hours sessions is on "one session" (i.e. the agent manages its own context via trimming and memory files, etc). What the project I linked did was "run in a bash loop", that basically resets the context every time the agent "finishes".

That would mean that every few hours the agent starts fresh, does the inspect repo thing, does the plan for that session, and so on. That would explain why it took it ~3 months to do what a human + ai could probably do in a few weeks. That's why it doesn't sound too ludicrous for me. If you look at the repo there are a lot of things that are not strictly needed for the initial prompt (make a programming language like go but with genz stuff, nocap).

Oh, and if you look at their discord + repo, lots of things don't actually work. Some examples do, some segfault. That's exactly what you'd expect from "running an agent in a loop". I still think it's impressive nonetheless.

The fact that you are so incredulous (and I get why that is, scepticism is warranted in this space) is actually funny. We are on the right track.

Re: Two things LLM coding agents are still bad at

#306

Post by laterium »

jsjshxnsbs wrote: post #103
ivape wrote: post #90

Earlier quoted context omitted.

[flagged]

I'll take the L when llms can actually do my job to the level I expect. Llms can do some of my work but they are tiring they make mistakes and they absolutely get confused by a sufficiently complex and large codebase. Quite frankly, not being able to discuss the pros and the cons of a technology with other engineers absolutely hinders innovation. A lot of discoveries come out of mistakes. Stop being so small minded.

Why is the bar for it to do your job or completely replace you? It's a tool. If it makes you 5% better at your job, then great. There's a recent study showing it has 15-20% productivity benefits: not completely useless, not 10x. I hope we can have nuance in the conversation.

Re: Two things LLM coding agents are still bad at

#308

Post by LeeRLemonIII »

I think #1 is not that big of a deal, though it does create problems sometimes. #2 is though a big issue. Which is weird since the whole thing is built as a chat model it seems it would be a lot more efficient for the bot to ask the questions of what to build beyond it's assumptions. Generally this lack of back and forth reasoning leads to a lot of then badly generated code. I would hope in the future there is some level of graded response that tries to discern the real intent of the users request through a discussion, rather than going to the fastest code answer.

Re: Two things LLM coding agents are still bad at

#309

Post by laterium »

goalieca wrote: post #190
IanCal wrote: post #167

Earlier quoted context omitted.

They're moderately unreliable text copying machines if you need exact copying of long arbitrary strings. If that's what you want, don't use LLMs. I don't think they were ever really sold as that, and we have better tools for that. On the other hand, I've had them easily build useful code, answer questions and debug issues complex enough to escape good engineers for at least several hours. Depends what you want. They'…

> I don't think they were ever really sold as that, and we have better tools for that. We have OpenAI calling gpt5 as having PhD level of intelligence and others like Anthropoc saying it will write all our code within months. Some are claiming it’s already writing 70%. I say they are being sold as a magical do everything tool.

They are lying, because their salary depends on them lying about it. Why does it even matter what they're saying? Why don't we listen to scientists, researchers, practicioners and the real users of the technology and stop repeating what the CEOs are saying?

The things they're saying are technically correct, the best kind of correct. The models beat human PhDs on certain benchmarks of knowledge and reasoning. They may write 70% of the easiest code in some specific scenario. It doesn't matter. They're useful tools that can make you slightly more productive. That's it.

When you see on tv that 9 out of 10 dentists recommend a toothpaste what do you do? Do you claim that brushing your teeth is a useless hype that's being pushed by big-tooth because they're exaggerating or misrepresenting what that means?

Re: Two things LLM coding agents are still bad at

#310

Post by moomoo11 »

FitchApps wrote: post #238
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

Reminds me when I asked Claude (through Windsurf) to create a S3 Lambda trigger to resize images (as soon as PNG image appears in S3, resize it). The code looked flawless and I deployed ..only to learn that I introduced a perpetual loop :) For every image resized, a new one would be created and resized. In 5 min, the trigger created hundreds of thousands of images ...what a joy was to clean that up in S3

Do you write tests and do local testing?
Post reply on HN