Live data from Hacker News

2025: The Year in LLMs

simonwillison.net

351–360 of 643 posts

Re: 2025: The Year in LLMs

#351

Speaking of asynchronous agents, what do people use? Claude Code for web is extremely limited, because you have no custom tools. Claude Code in GitHub Actions is vastly more useful, due to the custom environment, but ackward to use interactively. Are there any good alternatives?

What exactly do you mean by custom tools here? Just cli tools accessible to the agent?

Development environment needed to build and test the project.

Re: 2025: The Year in LLMs

#352
post #101

Speaking of new year and AI: my phone just suggested "Happy Birthday!" as the quick-reply to any "Happy New Year!" notification I got in the last hours. I'm not too worried about my job just yet.

It won't help to point out the worst examples. You're not competing with an outdated Apple LLM running on a phone. You're competing with Anthropic frontier models running on a multimillion dollar rack of servers.

Sounds like I'm much more affordable with better ROI

Re: 2025: The Year in LLMs

#353

Earlier quoted context omitted.

This is the problem, the entire internet is a really bad set of training data because it’s extremely polluted. Also the derived argument doesn’t really hold, just because you know about two things doesn’t mean you’d be able to come up with the third, it’s actually very hard most of the time and requires you to not do next token prediction.

The emergent phenomenon is that the LLM can separate truth from fiction when you give it a massive amount of data. It can figure the world out just as we can figure it out when we are as well inundated with bullshit data. The pathways exist in the LLM but it won’t necessarily reveal that to you unless you tune it with RL.

> The emergent phenomenon is that the LLM can separate truth from fiction when you give it a massive amount of data.

I don't believe they can. LLMs have no concept of truth.

What's likely is that the "truth" for many subjects is represented way more than fiction and when there is objective truth it's consistently represented in similar way. On the other hand there are many variations of "fiction" for the same subject.

Re: 2025: The Year in LLMs

#354
post #228

All these improvement in a single year, 2025. While this may seem obvious to those who follows along the AI / LLM news. It may be worth pointing out again ChatGPT was introduced to us in November 2022. I still dont believe AGI, ASI or Whatever AI will take over human in short period of time say 10 - 20 years. But it is hard to argue against the value of current AI, which many of the vocal critics on HN seems to have…

This is not a great argument:

> But it is hard to argue against the value of current AI [...] it is getting $1B dollar runway already.

The psychic services industry makes over $2 billion a year in the US [1], with about a quarter of the population being actual believers. [2].

[1] The https://www.ibisworld.com/united-states/industry/psychic-ser...

[2] https://news.gallup.com/poll/692738/paranormal-phenomena-met...

Re: 2025: The Year in LLMs

#355

Earlier quoted context omitted.

> You really owe it to yourself to try them out. I've worked at multiple AI startups in lead AI Engineering roles, both working on deploying user facing LLM products and working on the research end of LLMs. I've done collaborative projects and demos with a pretty wide range of big names in this space (but don't want to doxx myself too aggressively), have had my LLM work cited on HN multiple times, have LLM based gith…

Seriously, all that familiarity and you think an LLM "literally" can't invent anything that didn't already exist? Like, I'm sorry, but you're just flat-out wrong and I've got the proof sitting on my hard drive. I use this supposedly impossible program daily.

When a computer is able to invent things, we’ve achieved AGI. Do you believe we are already in the AGI era, or is the inventor in this case actually you?

Re: 2025: The Year in LLMs

#356
post #228

All these improvement in a single year, 2025. While this may seem obvious to those who follows along the AI / LLM news. It may be worth pointing out again ChatGPT was introduced to us in November 2022. I still dont believe AGI, ASI or Whatever AI will take over human in short period of time say 10 - 20 years. But it is hard to argue against the value of current AI, which many of the vocal critics on HN seems to have…

This is not a great argument: > But it is hard to argue against the value of current AI [...] it is getting $1B dollar runway already. The psychic services industry makes over $2 billion a year in the US [1], with about a quarter of the population being actual believers. [2]. [1] The https://www.ibisworld.com/united-states/industry/psychic-ser... [2] https://news.gallup.com/poll/692738/paranormal-phenomena-met...

What if these provide actual value through placebo-effect?

Re: 2025: The Year in LLMs

#357

Earlier quoted context omitted.

That's all very impressive, to be sure. But are you sure you're getting the point? As of 2025, LLMs are now very good at writing new code, creating new imagery, and writing original text. They continue to improve at a remarkable rate. They are helping their users create things that didn't exist before. Additionally, they are now very good at searching and utilizing web resources that didn't exist at training time. So…

> They are helping their users create things that didn't exist before. That is a derived output. That isn't new as in: novel. It may be unique but it is derived from training data. LLMs legitimately cannot think and thus they cannot create in that way.

I will find this often-repeated argument compelling only when someone can prove to me that the human mind works in a way that isn't 'combining stuff it learned in the past'.

5 years ago a typical argument against AGI was that computers would never be able to think because "real thinking" involved mastery of language which was something clearly beyond what computers would ever be able to do. The implication was that there was some magic sauce that human brains had that couldn't be replicated in silicon (by us). That 'facility with language' argument has clearly fallen apart over the last 3 years and been replaced with what appears to be a different magic sauce comprised of the phrases 'not really thinking' and the whole 'just repeating what it's heard/parrot' argument.

I don't think LLM's think or will reach AGI through scaling and I'm skeptical we're particularly close to AGI in any form. But I feel like it's a matter of incremental steps. There isn't some magic chasm that needs to be crossed. When we get there I think we will look back and see that 'legitimately thinking' wasn't anything magic. We'll look at AGI and instead of saying "isn't it amazing computers can do this" we'll say "wow, was that all there is to thinking like a human".

Re: 2025: The Year in LLMs

#358
post #296
post #180

Earlier quoted context omitted.

Yeah, I don't think I was particularly clear in that section. I don't think MCP is going to go away, but I do think it's unlikely to ever achieve the level of excitement it had in early 2025 again. If you're not building inside a code execution environment it's a very good option for plugging tools into LLMs, especially across different systems that support the same standard. But code execution environments are so mu…

I disagree. MCP will remain the best way to do most things for the same reason REST APIs are the main way to access non local services: they provide a way to secure and audit access to systems in a way that a coding environment cannot. And you can authorize actions depending on the well defined inputs and outputs. You can’t do that using just a bash script unless said script actually does SSO and calls REST APIs but…

I find it very hard to pick winners and losers in this environment where everything changes so quickly. Right now a lot of people are using bash as a glue environment for agents, even if they are not for developers.

Re: 2025: The Year in LLMs

#359

Earlier quoted context omitted.

This is not a great argument: > But it is hard to argue against the value of current AI [...] it is getting $1B dollar runway already. The psychic services industry makes over $2 billion a year in the US [1], with about a quarter of the population being actual believers. [2]. [1] The https://www.ibisworld.com/united-states/industry/psychic-ser... [2] https://news.gallup.com/poll/692738/paranormal-phenomena-met...

What if these provide actual value through placebo-effect?

You talking about psychics or LLMs?

Re: 2025: The Year in LLMs

#360

Earlier quoted context omitted.

> I don't understand why Hacker News is so dismissive about the coming of LLMs I find LLMs incredibly useful, but if you were following along the last few years the promise was for “exponential progress” with a teaser world destroying super intelligence. We objectively are not on that path. There is no “coming of LLMs”. We might get some incremental improvement, but we’re very clearly seeing sigmoid progress. I can’t…

> exponential progress First you need to define what it means. What's the metric? Otherwise it's very much something you can argue about.

Time spent being human and enjoying life.

I can’t point at many problems it has meaningfully solved for me. I mean real problems , not tasks that I have to do for my employer. It seems like it just made parts of my existence more miserable, poisoned many of the things I love, and generally made the future feel a lot less certain.

Post reply on HN