Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two things LLM coding agents are still bad at

kix.dev

Post reply on HN
Print view View original

261–270 of 382 posts

Re: Two things LLM coding agents are still bad at

#261

Post by sarchertech »

NitpickLawyer wrote: post #193
Gazoche wrote: post #188

Earlier quoted context omitted.

> written by cc "in a loop" in ~3 months? What does that mean exactly? I assume the LLM was not left alone with its task for 3 months without human supervision.

From the FAQ: > the following prompt was issued into a coding agent: > Hey, can you make me a programming language like Golang but all the lexical keywords are swapped so they're Gen Z slang? > and then the coding agent was left running AFK for months in a bash loop

I don’t buy it at all. Not even Anthropic or Open AI have come anywhere close to something like this.

Running for 3 months and generating a working project this large with no human intervention is so far outside of the capabilities of any agent/LLM system demonstrated by anyone else that the mostly likely explanation is that the promoter is lying about it running on its own for 3 months.

I looked through the videos listed as “facts” to support the claims and I don’t see anything longer than a few hours.

Re: Two things LLM coding agents are still bad at

#262

Post by fwip »

ivape wrote: post #53
rossant wrote: post #37

Recently, I asked Codex CLI to refactor some HTML files. It didn't literally copy and pasted snippets here and there as I would have done myself, it rewrote them from memory, removing comments in the process. There was a section with 40 successive links with complex URLs. A few days later, just before deployment to production, I wanted to double check all 40 links. First one worked. Second one worked. Third one worke…

You’re just not using LLMs enough. You can never trust the LLM to generate a url, and this was known over two years ago. It takes one token hallucination to fuck up a url. It’s very good at a fuzzy great answer, not a precise one. You have to really use this thing all the time and pick up on stuff like that.

I think part of the issue is that it doesn't "feel" like the LLM is generating a URL, because that's not what a human would be doing. A human would be cut & pasting the URLs, or editing the code around them - not retyping them from scratch.

Edit: I think I'm just regurgitating the article here.

Re: Two things LLM coding agents are still bad at

#263

Post by IanCal »

mbesto wrote: post #254
IanCal wrote: post #162

Earlier quoted context omitted.

They easily check a bunch of those boxes. > why don´t we stop pretending that we as users are stupid and don´t know how to use them This is in response to someone who saw a bunch of URLs coming out of it and was surprised at a bunch of them being wrong. That's using the tool wrong . It's like being surprised that the top results in google/app store/play store aren't necessarily the best match for your query but actua…

> This is in response to someone who saw a bunch of URLs coming out of it and was surprised at a bunch of them being wrong. That's using the tool wrong. It's like being surprised that the top results in google/app store/play store aren't necessarily the best match for your query but actually adverts! The CEO of Anthropic said I can fire all of my developers soon. How could one possibly be using the tool wrong? /s

If you base all your tech workings on the promises of CEOs you'll fail badly, you should not be surprised by this.

Re: Two things LLM coding agents are still bad at

#264

Post by fwip »

dncornholio wrote: post #204
quietbritishjim wrote: post #42

Earlier quoted context omitted.

Yesterday, I got Claude Code to make a script that tried out different point clustering algorithms and visualise them. It made the odd mistake, which it then corrected with help, but broadly speaking it was amazing. It would've taken me at least a week to write by hsnd, maybe longer. It was writing the algorithms itself, definitely not just simple CRUD stuff.

That's actually a very specific domain, which is well documented and researched in which LLM's will alawys do well. Shit will hit the fans quickly when you're going to do integration where it won't have a specific problem domain.

Yep - visualizing clustering algorithms is just the "CRUD app" of a different speciality.

One rule of thumb I use, is if you could expect to find a student on a college campus to do a task for you, an LLM will probably be able to do a decent job. My thinking is because we have a lot of teaching resources available for how to do that task, which the training has of course ingested.

Re: Two things LLM coding agents are still bad at

#265

Post by IanCal »

parineum wrote: post #252
buildbot wrote: post #227

Earlier quoted context omitted.

Would you hire a PhD to copy URLs by hand? Would them having PhD make it less likely they’d make a mistake than an high school student doing the same?

A high school student would use copy/paste and the urls would be perfect duplicates..

> A high school student would use copy/paste and the urls would be perfect duplicates..

Did the LLM have this?

Re: Two things LLM coding agents are still bad at

#266

Post by IanCal »

goalieca wrote: post #251
buildbot wrote: post #227

Earlier quoted context omitted.

Would you hire a PhD to copy URLs by hand? Would them having PhD make it less likely they’d make a mistake than an high school student doing the same?

Grad students and even post docs often do a lot of this manual labour for data entry and formatting. Been there, done that.

Manual data entry has lots of errors. All good workflows around this base themselves on this fact.

Re: Two things LLM coding agents are still bad at

#267

Post by hitarpetar »

buildbot wrote: post #227
goalieca wrote: post #190

Earlier quoted context omitted.

> I don't think they were ever really sold as that, and we have better tools for that. We have OpenAI calling gpt5 as having PhD level of intelligence and others like Anthropoc saying it will write all our code within months. Some are claiming it’s already writing 70%. I say they are being sold as a magical do everything tool.

Would you hire a PhD to copy URLs by hand? Would them having PhD make it less likely they’d make a mistake than an high school student doing the same?

I would not hire anyone for a role that requires computer use who does not know how to use copy/paste

Re: Two things LLM coding agents are still bad at

#268

Post by radarsat1 »

fwip wrote: post #239
Romario77 wrote: post #222

Earlier quoted context omitted.

LLMs are not good at "cycles" - when you have to go over a list and do the same action on each item. It's like it has ADHD and forgets or gets distracted in the middle. And the reason for that is that LLMs don't have memory and process the tokens, so as they keep going over the list the context becomes bigger with more irrelevant information and they can lose the reason they are doing what they are doing.

It would be nice if the tools we usually use for LLMs had a bit more programmability. In this example, It we could imagine being able to chunk up work by processing a few items, then reverting to a previous saved LLM checkpoint of state, and repeating until the list is complete. I imagine that the cost of saving & loading the current state must be prohibitively high for this to be a normal pattern, though.

Agreed. You basically want an LLM to have a tool that writes its own agent to accomplish a repetitive task. I think this is doable.

Re: Two things LLM coding agents are still bad at

#269

Post by jeswin »

tom_ wrote: post #191
jeswin wrote: post #113

Earlier quoted context omitted.

Very true. However, to claim that the "API looks completely different for Postgre and SQLite" is disingenuous. What was he looking at?

There are two examples on the landing page, and they both look quite different. Surely if the API is the same for both, there'd be just one example that covers both cases, or two examples would be deliberately made as identical as possible? (Like, just a different new somewhere, or different import directive at the top, and everything else exactly the same?) I think that's the point. Perhaps experienced users of rele…

If you're mentioning the first two examples, they're doing different things. The pg example does an orderby, and the sqlite example does a join. You'll be able to switch the client (ie, better-sqlite and pg-promise) in either statement, and the same query would work on the other database.

Maybe I should use the same example repeated for clarity. Let me do that.

Edit: Fixed. Thank you.

Re: Two things LLM coding agents are still bad at

#270

Post by sarchertech »

freetonik wrote: post #86
NitpickLawyer wrote: post #52

Earlier quoted context omitted.

> almost always the first group presents examples of simple CRUD apps How about a full programming language written by cc "in a loop" in ~3 months? With a compiler and stuff? https://cursed-lang.org/ It might be a meme project, but it's still impressive as hell we're here. I learned about this from a yt content creator that took that repo, asked cc to "make it so that variables can be emojis", and cc did that 5$ late…

Ok, not trivial for sure, but not novel? IIUC, the language does not have really new concepts, apart from the keywords (which is trivial). Impressive nonetheless.

There’s no evidence that this ever happened other than this guy’s word. And since the claim that he ran an agent with no human intervention for 3 months is so far outside of any capabilities demonstrated by anyone else, I’m going to need to see some serious evidence before I believe it.
Post reply on HN