Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2025: The Year in LLMs

simonwillison.net

Post reply on HN
Print view View original

521–530 of 643 posts

Re: 2025: The Year in LLMs

#521

Post by HumblyTossed »

aspenmartin wrote: post #474
HumblyTossed wrote: post #446

Earlier quoted context omitted.

It's a great tool, but right now it's only being used to feed the greed. >> Again, I guess no one knew AI would be as big as it is today, and it is only just started. People have been saying similar about self driving cars for years now. "AI" is another one of those expensive ideas that we'll get 85% of the way there and then to get the other 15% will be way more expensive than anyone will want to pay for. It's alrea…

I agree -- skepticism is totally healthy. And there are so many great ways to poke holes in the true underlying narratives (not the headlines that people seem to pull from). E.g. evaluation science is a wasteland (not for wont of very smart people trying very hard to get them right). How do we tackle the power requirements in a way that is sustainable? Etc. etc. But stuff like this im not sure I understand: > It's a…

>> if its a great tool, then how is it _only_ being used to "feed the greed" and what do you mean by that?

Look around?

Re: 2025: The Year in LLMs

#522

Post by aspenmartin »

HumblyTossed wrote: post #521
aspenmartin wrote: post #474

Earlier quoted context omitted.

I agree -- skepticism is totally healthy. And there are so many great ways to poke holes in the true underlying narratives (not the headlines that people seem to pull from). E.g. evaluation science is a wasteland (not for wont of very smart people trying very hard to get them right). How do we tackle the power requirements in a way that is sustainable? Etc. etc. But stuff like this im not sure I understand: > It's a…

>> if its a great tool, then how is it _only_ being used to "feed the greed" and what do you mean by that? Look around?

Very confused, I still don’t know what you mean at all

Re: 2025: The Year in LLMs

#523

Post by aspenmartin »

bopbopbop7 wrote: post #517
aspenmartin wrote: post #512

Earlier quoted context omitted.

I could have guessed you would say that :) but METR is not an unbiased study either. Maybe you mean that METR is less likely to intentionally inflate their numbers? If you insist or believe in a conspiracy I don’t think there’s really anything I or others will be able to say or show you that would assuage you, all I can say is I’ve seen the raw data. It’s a mess and again we’re stuck with proxies (which are bad since…

I bet it does reduce your cognitive load, considering you, in your own words "Give up when Claude is hopelessly lost". No better way to reduce cognitive load.

I give up using Claude when it gets hopelessly lost, and then my cognitive load increases.

Re: 2025: The Year in LLMs

#524

Post by pjc50 »

simonw wrote: post #463
pjc50 wrote: post #456

Earlier quoted context omitted.

Did Uber actually do a lot of capital investment? They don't own the cars, for example.

I believe they spent a huge amount of money on incentives to help sign up drivers, and discounts to help attract customers.

Yes, but that's loss leader rather than capital investment. You can't put a customer on the balance sheet and depreciate them. Once you've paid for a free ride, you own nothing tangible.

Re: 2025: The Year in LLMs

#525

Post by CamperBob2 »

windexh8er wrote: post #498
CamperBob2 wrote: post #462

Earlier quoted context omitted.

There's no such thing as people without language, except for infants and those who are so mentally incapacitated that the answer is self-evidently "No, they cannot." Language is the substrate of reason. It doesn't need to be spoken or written, but it's a necessary and (as it turns out) sufficient component of thought.

There are quite a few studies to refute this highly ignorant comment. I'd suggest some reading [0]. From the abstract: "Is thought possible without language? Individuals with global aphasia, who have almost no ability to understand or produce language, provide a powerful opportunity to find out. Astonishingly, despite their near-total loss of language, these individuals are nonetheless able to add and subtract, solve…

Yeah, you can prove pretty much anything with a pubmed link. Do dead salmon "think?" fMRI says maybe!

https://pmc.ncbi.nlm.nih.gov/articles/PMC2799957/

The resources that the brain is using to think -- whatever resources those are -- are language-based. Otherwise there would be no way to communicate with the test subjects. "Language" doesn't just imply written and spoken text, as these researchers seem to assume.

Re: 2025: The Year in LLMs

#526

Post by senordevnyc »

qualifck wrote: post #490
senordevnyc wrote: post #449

Earlier quoted context omitted.

I’ve been reading this comment multiple times a week for the last couple years. Constant assertions that we’re starting to hit limits, plateau, etc. But a cursory glance at where we are today vs a year ago, let alone two years ago, makes it wildly obvious that this is bullshit. The pace of improvement of both models and tooling has been breathtaking. I could give a shit whether you think it’s “exponential”, people li…

People keep saying stuff like this. That the improvements are so obvious and breathtaking and astronomical and then I go check out the frontier LLMs again and they're maybe a tiny bit better than they were last year but I can't actually be sure bcuz it's hard to tell. sometimes it seems like people are just living in another timeline.

I’m genuinely curious what your “checking the frontier LLMs” looks like, especially if you haven’t used AI since last year.

Re: 2025: The Year in LLMs

#527

Post by shimman »

insin wrote: post #501
aspenmartin wrote: post #478

Earlier quoted context omitted.

Sure: at my MAANG company, where I watch the data closely on adoption of CC and other internal coding agent tools, most (significant) LOC are written by agents, and most employees have adopted coding agents as WAU, and the adoption rate is positively correlated with seniority. Like a lot of things LLM related (Simon Willison's pelican test, researchers + product leaders implementing AI features) I also heavily "vibe"…

> - Give up when Claude is hopelessly lost You love to see "Maybe completely waste my time" as part of the normal flow for a productivity tool

There's a reason why sunk cost IS a fallacy and not a sound strategy.

Re: 2025: The Year in LLMs

#528

Post by shimman »

pjc50 wrote: post #456
steveBK123 wrote: post #362

Earlier quoted context omitted.

Indeed, its the old Uber playbook at nearly two extra orders of magnitude. It is a large enough number to simply run out of private capital to consume before it turns cash flow positive. Lots of things sell well if sold at such a loss. I’d take a new Ferrari for $2500 if it was on offer.

Did Uber actually do a lot of capital investment? They don't own the cars, for example.

Uber nakedly broke the law and beat down labor, I'm honestly shocked none of the executives went to prison.

Re: 2025: The Year in LLMs

#529

Post by CamperBob2 »

windexh8er wrote: post #506
CamperBob2 wrote: post #408

Earlier quoted context omitted.

Ask an LLM what's the difference between a human brain and an LLM. If an LLM could "think" it wouldn't give you the answer it just did. I imagine that sounded more profound when you wrote it than it did just now, when I read it. Can you be a little more specific, with regard to what features you would expect to differ between LLM and human responses to such a question? Right now, LLM system prompts are strongly geare…

> Right now, LLM system prompts are strongly geared towards not claiming that they are humans or simulations of humans. If your point is that a hypothetical "thinking" LLM would claim to be a human, that could certainly be arranged with an appropriate system prompt. You wouldn't know whether you were talking to an LLM or a human -- just as you don't now -- but nothing would be proved either way. That's ultimately why…

How does it know when it’s wrong without an external verifier? If the training data says X and the answer is Y - how will it ever know it was wrong and reach the correct conclusion?

You need to read a few papers with publication dates after 2023.

Re: 2025: The Year in LLMs

#530

Post by CamperBob2 »

windexh8er wrote: post #497
closewith wrote: post #409

Earlier quoted context omitted.

You’re arguing against a straw man. No one is claiming LLMs have beliefs, intentions, or understanding. They don’t need them to be economically useful.

Oh yes, they are. And beyond people claiming that LLMs are basically sentient you have people like CamperBob2 who made this wild claim: """There's no such thing as people without language, except for infants and those who are so mentally incapacitated that the answer is self-evidently "No, they cannot." Language is the substrate of reason. It doesn't need to be spoken or written, but it's a necessary and (as it turns…

How'd they communicate with the test subjects?

That's "language."

Post reply on HN