Live data from Hacker News

2025: The Year in LLMs

simonwillison.net

461–470 of 643 posts

Re: 2025: The Year in LLMs

#461

Earlier quoted context omitted.

ChatGPT has roughly 800 million weekly active users. Almost everyone around me uses it daily. I think you are underestimating the adoption.

Usage plunges on the weekends and during the summer, suggesting that a significant portion of users are students using ChatGPT for free or at heavily subsidized rates to do homework (i.e., extremely basic work that is extraordinarily well-represented in the training data). That usage will almost certainly never be monetizable, and it suggests nothing about the trajectory of the technology’s capability or popularity.…

The summer slump was a thing in 2023 but apparently didn't repeat in 2024: https://www.similarweb.com/blog/insights/ai-news/chatgpt-bea...

The weekend slumps could equally suggest people are using it at work.

Re: 2025: The Year in LLMs

#462

Earlier quoted context omitted.

Mastery of words is thinking? That's the crazy thing. Yes, in fact, it turns out that language encodes and embodies reasoning. All you have to do is pile up enough of it in a high-dimensional space, use gradient descent to model its original structure, and add some feedback in the form of RL. At that point, reasoning is just a database problem, which we currently attack with attention. No one had the faintest clue. E…

So people without language cannot reason? I don't think so.

There's no such thing as people without language, except for infants and those who are so mentally incapacitated that the answer is self-evidently "No, they cannot."

Language is the substrate of reason. It doesn't need to be spoken or written, but it's a necessary and (as it turns out) sufficient component of thought.

Re: 2025: The Year in LLMs

#463
post #456

Earlier quoted context omitted.

Indeed, its the old Uber playbook at nearly two extra orders of magnitude. It is a large enough number to simply run out of private capital to consume before it turns cash flow positive. Lots of things sell well if sold at such a loss. I’d take a new Ferrari for $2500 if it was on offer.

Did Uber actually do a lot of capital investment? They don't own the cars, for example.

I believe they spent a huge amount of money on incentives to help sign up drivers, and discounts to help attract customers.

Re: 2025: The Year in LLMs

#464
I predict 2026 will be the year of the first AI Agent "worm" (or virus?). Kind of like the Morris worm running amok as an experiment gone wrong, I think we will sometime soon have someone set up an AI agent whose core loop is to try to propagate itself, either as an experiment or just for the lulz.

The actual Agent payload would be very small, likely just a few hundred line harness plus system prompt. It's just a question of whether the agent will be skilled enough to find vulnerabilities to propagate. The interesting thing about an AI worm is that it can use different tricks on different hosts as it explores its own environment.

If a pure agent worm isn't capable enough, I could see someone embedding it on top of a more traditional virus. The normal virus would propagate as usual, but it would also run an agent to explore the system for things to extract or attack, and to find easy additional targets on the same internal network.

A main difference here is that the agents have to call out to a big SotA model somewhere. I imagine the first worm will simply use Opus or ChatGPT with an acquired key, and part of it will be trying to identify (or generate) new keys as it spreads.

Ultimately, I think this worm will be shut down by the model vendor, but it will have to have made a big enough splash beforehand to catch their attention and create a team to identify and block keys making certain kinds of requests.

I'd hope OpenAI, Anthropic, etc have a team and process in place already to identify suspicious keys, eg, those used from a huge variety of IPs, but I wouldn't be surprised if this were low on their list of priorities (until something like this hits).

Re: 2025: The Year in LLMs

#465

The difference between the performance of models between 2024 and 2025 has been so stark, that graph really shows it. There are still many people on these forums who seem to think AI’s produce terrible code unless ultra supervised, and I can’t help but suspect some of them tried it a little while ago and just don’t understand how different it is now compared to even quite recently.

I used Gemini Pro, Claude Pro yesterday a couple of dozen times and basically have been daily.

I have a project to convert my multiplayer XNA game from C# to Javascript and to add networking to the game-play using LLMs.

They are far worse at it now than they were a year ago. They actually implemented the requirements (Though inaccurately) to the best of their ability a year ago. Especially Gemini.

Now they don't even come remotely close to implementing just the basic requirements.

The thing is, I'm giving them the entirety of the C# source code and spelling out what they should do.

Re: 2025: The Year in LLMs

#466

Earlier quoted context omitted.

The question is how rapid the adoption is. The price of failure in the real world is much higher ($$$, environmental, physical risks) vs just "rebuild/regenerate" in the digital realm.

Military adoption is probably a decent proxy indicator - and they are ready to hand the kill switch to autonomous robots

Maybe. There the cost of failure again is low. Its easier to destroy than to create. Economic disruption to workers will take a bit longer I think.

Don't get me wrong; I hope that we do see it in physical work as well. There is more value to society there; and consists of work that is risky and/or hard to do - and is usually needed (food, shelter, etc). It also means that the disruption is an "everyone" problem rather than something that just affects those "intellectual" types.

Re: 2025: The Year in LLMs

#467
post #414

Earlier quoted context omitted.

2022/2023: "It hallucinates, it's a toy, it's useless." 2024/2025: "Okay, it works, but it produces security vulnerabilities and makes junior devs lazy." 2026 (Current): "It is literally the same thing as a psychic scam." Can we at least make predictions for 2027? What shall the cope be then! Lemme go ask my psychic.

2022/2023: "Next year software engineering is dead" 2024: "Now this time for real, software engineering is dead in 6 months, AI CEO said so" 2025: "I know a guy who knows a guy who built a startup with an LLM in 3 hours, software engineering is dead next year!" What will be the cope for you this year?

The cope + disappointment will be knowing that a large population of HN users will paint a weird alternative reality. There are a multitude of messages about AI that are out there, some are highly detached from reality (on the optimistic and pessimistic side). And then there is the rational middle, professionals who see the obvious value of coding agents in their workflow and use them extensively (or figure out how to best leverage them to get the most mileage). I don't see software engineering being "dead" ever, but the nature of the job _has already changed_ and will continue to change. Look at Sonnet 3.5 -> 3.7 -> 4.5 -> Opus 4.5; that was 17 months of development and the leaps in performance are quite impressive. You then have massive hardware buildouts and improvements to stack + a ton of R&D + competition to squeeze the juice out of the current paradigm (there are 4 orders of magnitude of scaling left before we hit real bottlenecks) and also push towards the next paradigm to solve things like continual learning. Some folks have opted not to use coding agents (and some folks like yourself seem to revel in strawmanning people who point out their demonstrable usefulness). Not using coding agents in Jan 2026 is defensible. It won't be defensible for long.

Re: 2025: The Year in LLMs

#468

The difference between the performance of models between 2024 and 2025 has been so stark, that graph really shows it. There are still many people on these forums who seem to think AI’s produce terrible code unless ultra supervised, and I can’t help but suspect some of them tried it a little while ago and just don’t understand how different it is now compared to even quite recently.

I used Gemini Pro, Claude Pro yesterday a couple of dozen times and basically have been daily. I have a project to convert my multiplayer XNA game from C# to Javascript and to add networking to the game-play using LLMs. They are far worse at it now than they were a year ago. They actually implemented the requirements (Though inaccurately) to the best of their ability a year ago. Especially Gemini. Now they don't even…

Weird. I would expect Gemini 3 Pro and Claude Opus 4.5 to run rings around Gemini 1.5 Pro and Claude Sonnet 3.5.

How are you running them - regular chat interface or do you have them setup with Claude Code or Gemini CLI?

Re: 2025: The Year in LLMs

#469
post #424

Earlier quoted context omitted.

His comment is far better than the rampant astroturfing from stakeholders going on everywhere on this website that is being mitigated not at all whatsoever. There is a wealth of information present suggesting these things are so bad for everyone in so many ways.

What are some specific links to the rampant astroturfing that you feel is going on on this website and which https://news.ycombinator.com/item?id=46450296 is better than?

Let's see, going off of just top-level comments in this thread alone:

didip, timonoko, mark_I_watson, icapybara, _pdp_, agentifysh, sanreau,

There's no way to know if these are genuine thoughts or incentivized compelled speech.

nativeit has a good way of putting it.

Your replies to "anonnon" make me less than hopeful for the future of HN in regards to AI. Seems like this might be trending in the direction of Reddit, where the interests are basically all paid for and imposed rather than being genuine and organic, and dissent is aggressively shut out.

"Curious conversation" does not really apply when it is compelled via monetary interest without any consideration toward potentially serious side effects.

"At least when herding cats, you can be sure that if the cats are hungry, they will try to get where the food is." This part of the guy's comment is actually funny and apt. Somehow that escaped you when you wrote your threat reply. That makes me wonder how mind-controlled you are.

"yupyupyups" has a small summary of some of the negatives, yet is being flagged. "techpression" similarly does, though is a bit more negative in his remarks. Also being flagged.

So the whole thread reads like this: 1.) talking about benefits? bubble to the top 2.) criticize? Either threatened by Dang or flagged to the bottom

Sounds a whole lot like compelled speech to me. Sounds a whole lot like mind-control.

It's pretty sad to see really.

It might just be your rule system. I personally want to see criticism. I don't have the sensitivity you have toward personal attacks or what you "deem" personal attacks when it is text on-screen. I don't care. I want to see what useful information might come out of it. I think your policing just makes everything worse to be honest. The thread will just die out in a day anyway.

I think I have criticized it in the past and you or some other staff said that it's a slippery slope toward useless aggressive banter that derails topics, but I don't know. I really don't agree with it. That's just my life experience.

Reddit is kind of like this. And it's basically turned into imposed topics rather than organic topics with massive amounts of echo-chambering in each delusional sub-reddit. Anything remotely against the grain is harshly culled as soon as possible. You can only imagine what the back-end looks like for that kind of thing. Money being involved at many steps is guaranteed.

And yeah as another commenter pointed out, this one guy's blog being at the top of hacker news every time is potentially suspicious as well.

I think I originally came to this place more than Reddit 10+ years ago because yeah it felt like people just excited and curious about their tech topics and it didn't feel like it was being rampantly policed or pushing a political agenda etc. I guess I should just not participate in these threads because the topic is tired on me at this point.

Wait I just read your user page and this is actually hilarious:

"Conflict is essential to human life, whether between different aspects of oneself, between oneself and the environment, between different individuals or between different groups. It follows that the aim of healthy living is not the direct elimination of conflict, which is possible only by forcible suppression of one or other of its antagonistic components, but the toleration of it—the capacity to bear the tensions of doubt and of unsatisfied need and the willingness to hold judgement in suspense until finer and finer solutions can be discovered which integrate more and more the claims of both sides. It is the psychologist's job to make possible the acceptance of such an idea so that the richness of the varieties of experience, whether within the unit of the single personality or in the wider unit of the group, can come to expression."

Marion Milner, 'The Toleration of Conflict', Occupational Psychology, 17, 1, January 1943

This made me immediately and uncontrollably guffaw.

Re: 2025: The Year in LLMs

#470
post #228

All these improvement in a single year, 2025. While this may seem obvious to those who follows along the AI / LLM news. It may be worth pointing out again ChatGPT was introduced to us in November 2022. I still dont believe AGI, ASI or Whatever AI will take over human in short period of time say 10 - 20 years. But it is hard to argue against the value of current AI, which many of the vocal critics on HN seems to have…

> All these improvement in a single year > hard to argue against the value of current AI > People are willing to pay $200 per month, and it is getting $1B dollar runway already. Those are 3 different things. There can be a LOT of fast and significant improvements but still remain extremely far from the actual goal, so far it looks like actually little progress. People pay for a lot of things, including snake oil, so…

> People pay for a lot of things, including snake oil, so convincing a lot of people to pay a bit is not in itself a proof of value

But do you really believe e.g. Claude code is snake oil? I pay $200 / month for Claude, which is something I would have thought monumentally insane maybe 1-2 years ago (e.g. when ChatGPT came out with their premium subscription price I thought that seemed so out of touch). I don't think we would be seeing the subscription rates and the retention numbers if it really was snake oil.

> Finally yes, $1B is a LOT of money for you and I... but for the largest corporations it's actually not a lot. For reference Google earned that in revenue... per day in 2023. Anyway that's still a big number BUT it still has to be compared with, well how much does OpenAI burn. I don't have any public number on that but I believe the consensus is that it's a lot. So until we know that number we can't talk about an actual runway.

this gets brought up a lot but I'm not sure I understand why folks on a forum called YCombinator, a startup accelerator, would make this sound like an obvious sign of charlatanism; operating at a loss is nothing new and anthropic / openAI strategy seems perfectly rational: they are scaling and capturing market share, and TAM is insane.

Post reply on HN