Live data from Hacker News

The last six months in LLMs in five minutes

simonwillison.net

451–460 of 631 posts

Re: The last six months in LLMs in five minutes

#451

Earlier quoted context omitted.

Extrapolating the final position of the goalposts

That's a halting problem, I think.

And Claude can actually tackle it the same way as humans do - here in the real world, where we don't have time to let some nonsense like "mathematically proven to be unsolvable" to stand between us and our goals: it can eyeball the code and give a good enough guess.

Re: The last six months in LLMs in five minutes

#452
post #204

Earlier quoted context omitted.

Depends on how it’s done. I use it a lot now for knocking up grafana charts etc. It’s not so much that the LLM is feeding the numbers through. You can still use real tools to analyse and summarise the numbers, it’s just much quicker at driving them. As ever with data analysis, two things will continue to be true. Real insights come from spotting something that looks off and digging into it deeper. Secondly, it’s real…

Sure it depends how it is done but for most uses I'd say they are not appropriate - building tools with them is ok if you double check (though how many people will when the answers seem good enough at first?). I'd find it really troubling if financial analysts are using them without knowing the deep limitations of the tooling (which the companies selling them will not highlight for you). They don't actually count or…

Don’t get me wrong, I very much agree with the danger. As I highlighted - I saw it this morning when someone used Claude to draw the wrong conclusions.

I’m saying there is a way in which they can be used where there isn’t scope for numerical hallucinations at all. They can write sql queries, for example, without ever being allowed to even see numbers.

What invariably does and will happen though is they’ll inner join instead of left join and some data will get missed. Or there will be some missing context (users in this set already have a certain class of property by virtue of some selection bias and that will be mistreated as some signal etc).

Re: The last six months in LLMs in five minutes

#453

Earlier quoted context omitted.

Take it up with Anthropic. It's actually their billion-dollar TUI product you're commenting on. The problem with being such a naysayer is that you're entirely disconnected from what's going on. You haven't tried an agent like Claude Code and experienced it for yourself, so you don't recognise what it looks like when it's in front of you.

> Take it up with Anthropic. It's actually their billion-dollar TUI product you're commenting on. That's like blaming the company making hammers because you're unable to build a lasting house with the hammer, it really isn't up to Anthropic, but all about how you use the tool you're holding.

Do they also hold their hammer wrong when their TUI flickers for months?

Re: The last six months in LLMs in five minutes

#454

Earlier quoted context omitted.

Hey that's fine. You're free to make whatever judgment you wish. But I still stand by the quality of my code, including here. You and I don't need to agree. What decades of managing codebases (public and private, huge and small) has taught me is that there will always be an endless list of bugs and feature ideas and nice-to-haves and technical debt pressures in any given project. You'll never get to them all, so you…

> But I still stand by the quality of my code, including here. You and I don't need to agree. You aren’t having a disagreement with a person. You’re having a disagreement with reality.

> You aren’t having a disagreement with a person. You’re having a disagreement with reality.

How so? Are you going to instruct us all on how a termcaps mismatch bug is an indicator of poor code quality, rather than an unfortunate bug emerging from within the chaos of the many layers of disparate technologies that must somehow be stitched together (along with their idiosyncrasies) in order to make a project like this work?

Re: The last six months in LLMs in five minutes

#455

Last 6 months is humanity losing control of LLMs. - Memory market cornering which mitigated the adoption of local AI despite great open model being released. - Fast penetration of IP exfiltrating tools in companies world-wide. - Developers producing more code that they can read. - Autonomous agents killing Open Source by siphoning the attention economy - Autonomous agents destroyed online communities (including HN) -…

> - Memory market cornering (...)

Wait, what? What is that?

> - Fast penetration of IP exfiltrating tools in companies world-wide.

That goes on the benefit side, I believe.

> - Autonomous agents killing Open Source by siphoning the attention economy

Anything attention economy disappearing is a "good riddance" to me.

Re: The last six months in LLMs in five minutes

#456

Earlier quoted context omitted.

Have you seen the automated tests that QA members deliver? My experience is that they are horrible, and it's not so hard to beat that low quality bar with an LLM. I have a theory: if they were good at writing automated tests, they would have been developers instead of QA engineers. Not saying that there aren't any high quality QA engineers, I worked with some. But LLM's raised the bar in a way that most QA engineers…

Huh, never thought about QA writing unit tests. In my limited experience they write test cases, test each story, do regression test, verify bugs from customers. All by hand. At my current job I don't want to miss them.

They test everything manually and don't have any automated end-to-end tests? That basically proves my point ;).

Re: The last six months in LLMs in five minutes

#457

Earlier quoted context omitted.

This is a pretty wild take. What percentage of human engineers are creating novel solutions for hard problems, you think? I work in R&D and even my work is 90% doing things that other people already solved. If you are really doing cutting edge SOTA work that has never been done by another human in some form or another, then kudos to you and I want your job.

> What percentage of human engineers are creating novel solutions for hard problems, you think? IMO Every engineer should try spending his time in a company that tries to solve new problems. Otherwise we will be stuck, as we are now, with big tech paying you mountains of money for doing nothing, incentivizing you to embark on useless activities for letting other managers have a career, fear layoffs and when that happ…

Please you don’t need to stress anything. I think you are conflating ideas.

Unique game loops ideas make a good game, it has very little to do with the engineering. This is true for most software engineering products. Most engineering work is just reinventing or reimplementing existing ideas, what you describe rarely exists. It may exist in that the people learning the new ideas think it’s novel but very little is truly unique.

Re: The last six months in LLMs in five minutes

#458

> The coding agents got really good It's since november 2025, the so called "inflection point", that I'm still wondering for who coding agents become "really good". All I observe they got better at tool call and answering questions about big codebases, especially if the question has a vague pattern to search, and they're superuseful for that! For generating production code even with a lot of steering and baby sitting…

Steve Yegge wrote about this in his book Vibe Coding. He says it takes about a year of experience before you're consistently getting good results. He writes about lots of different techniques for doing that, but also says a lot of it comes down to just getting a feel for when the LLM is going to go haywire.

Re: The last six months in LLMs in five minutes

#459

> The coding agents got really good It's since november 2025, the so called "inflection point", that I'm still wondering for who coding agents become "really good". All I observe they got better at tool call and answering questions about big codebases, especially if the question has a vague pattern to search, and they're superuseful for that! For generating production code even with a lot of steering and baby sitting…

Which languages and subject matter do you work with?

c/c++, java, kotlin, go, some perl scripting, some javascript. Gaming industry

Re: The last six months in LLMs in five minutes

#460

Earlier quoted context omitted.

The duplication I'm seeing isn't just "same text repeated" but structural duplication. Doing a quick 5 minute look again just to give you some pointers; runtime.MountSpec construction in buildMounts, Workdir vs aux-dir mount-mode handling, repeated one-off mount append blocks, overlay detection and so on, the list goes on. Just those should account for 200+ lines. Look for slight variations of the same thing but with…

You consider adding in-place constructed items to an array to be code duplication?

I've noticed that the bar for "quality" when people judge AI is often significantly higher than what they'd hold a human to. I'm not saying GP et al are doing this (I haven't looked myself), but it is a widespread pattern I've noticed both professionally and personally. I don't know why it is.
Post reply on HN