Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2025: The Year in LLMs

simonwillison.net

Post reply on HN
Print view View original

601–610 of 643 posts

Re: 2025: The Year in LLMs

#601

Post by CamperBob2 »

emp17344 wrote: post #598
CamperBob2 wrote: post #525

Earlier quoted context omitted.

Yeah, you can prove pretty much anything with a pubmed link. Do dead salmon "think?" fMRI says maybe! https://pmc.ncbi.nlm.nih.gov/articles/PMC2799957/ The resources that the brain is using to think -- whatever resources those are -- are language-based. Otherwise there would be no way to communicate with the test subjects. "Language" doesn't just imply written and spoken text, as these researchers seem to assume.

There’s linguistic evidence that, while language influences thought, it does not determine thought - see the failure of the strong Sapir-Whorf hypothesis. This is one of the most widely studied and robust linguistic results - we actually know for a fact that language does not determine or define thought.

How's the replication rate in that field? Last I heard it was below 50%.

How can you think without tokens of some sort? That's half of the question that has to be answered by the linguists. The other half is that if language isn't necessary for reasoning, what is?

We now know that a conceptually-simple machine absolutely can reason with nothing but language as inputs for pretraining and subsequent reinforcement. We didn't know that before. The linguists (and the fMRI soothsayers) predicted none of this.

Re: 2025: The Year in LLMs

#602

Post by emp17344 »

CamperBob2 wrote: post #601
emp17344 wrote: post #598

Earlier quoted context omitted.

There’s linguistic evidence that, while language influences thought, it does not determine thought - see the failure of the strong Sapir-Whorf hypothesis. This is one of the most widely studied and robust linguistic results - we actually know for a fact that language does not determine or define thought.

How's the replication rate in that field? Last I heard it was below 50%. How can you think without tokens of some sort? That's half of the question that has to be answered by the linguists. The other half is that if language isn't necessary for reasoning, what is? We now know that a conceptually-simple machine absolutely can reason with nothing but language as inputs for pretraining and subsequent reinforcement. We d…

Read about linguistic history and make up your own mind, I guess. Or don’t, I don’t care. You’re dismissing a series of highly robust scientific results because they fail to validate your beliefs, which is highly irrational. I'm no longer interested in engaging with you.

Re: 2025: The Year in LLMs

#603

Post by ACCount37 »

HarHarVeryFunny wrote: post #571
ACCount37 wrote: post #560

Earlier quoted context omitted.

> which to be useful would require the "LLM" to then be autonomous and act in some (real/virtual) world in order to learn. You described modern RLVR for tasks like coding. Plug an LLM into a virtual env with a task. Drill it based on task completion. Force it to get better at problem-solving. It's still an autoregressive next token prediction engine. 100% LLM, zero architectural changes. We just moved it past pure im…

Yes, if all you did was replace current pre/mid/post training with a new (elusive holy grail) runtime continual learning algorithm, then it would definitely still just be a language model. You seem to be talking about it having TWO runtime continual learning algorithms, next-token and long-horizon RL, but of course RL is part of what we're calling an LLM. It's not obvious if you just did this without changing the lea…

No. There are no architectural changes and no "second runtime learning algorithm". There's just the good old in-context learning that all LLMs get from pre-training. RLVR is a training stage that pressures the LLM to take advantage of it on real tasks.

"Runtime continual learning algorithm" is an elusive target of questionable desirability - given that we already have in-context learning, and "get better at SFT and RLVR lmao" is relatively simple to pull off and gives kickass gains in the here and now.

I see no reason why "behave in a more human/animal-like self-motivated agentic fashion" can't be obtained from more RLVR, if that's what you want to train your LLMs for.

Re: 2025: The Year in LLMs

#604

Post by tantony »

Claude Opus 4.5 has been a big step up for me personally, and I used to think Sonnet 3.5 was good. It is an amazing deal at $20.

Just yesterday, it helped me parse out and understand a research paper - complete with step-by-step examples (this one: https://research.nvidia.com/sites/default/files/pubs/2016-03...). I will now go ahead and implement it myself, possibly relegating some of the more grunt-work type tasks to Claude code.

Without it, I would have been struggling through the paper for days, wading through WGSL shader code and there would be a high chance that I just give up on it since this is for a side project and not my $job.

It has been a major force multiplier just for learning things. I have had the $20 subscription for about a year now. I bump it up to the $100 plan if I happen to be working on some project that eats through the $20 allocation. This happens to be one such month. I will probably go back to the $20 plan after this month. I continue to get a lot of value out of it.

Re: 2025: The Year in LLMs

#605

Post by uxcolumbo »

scotty79 wrote: post #586
uxcolumbo wrote: post #252

Earlier quoted context omitted.

Will local folks get those jobs to build the data center? And if so, what happens to those builders once the data center is built?

> Will local folks get those jobs to build the data center? Yes. At some point the demand will be so high that imported workers won't suffice and local population will need to be trained and hired. > And if so, what happens to those builders once the data center is built? They are going to be moved to a new place where the datacenters will need to be built next. Mobility if the workforce was often cited as one of the…

So local people in town 1 who are getting these jobs to build the data center will then have to move to town 2 to build a data center there? What happens to the local people in town 2 who are also looking for construction jobs?

Re: 2025: The Year in LLMs

#606

Post by yencabulator »

jll29 wrote: post #349
odiroot wrote: post #318

Earlier quoted context omitted.

I'm very relieved we've moved away from rewriting everything in Rust.

There's no reason not to use Rust for LLM-generated code in the longer term (other than lack of Rust code to learn from in the shorter term). The stricter typing of Rust would make sematic errors in generated code come out more quickly than in e.g. Python because using static typing the chances are that some of the semantic errors are also type violations.

> (other than lack of Rust code to learn from in the shorter term)

FWIW Claude Code is quite good at writing Rust, and Claude and Gemini are both surprisingly good at explaining complex third-party Rust libraries.

Re: 2025: The Year in LLMs

#607

Post by ben_w »

bopbopbop7 wrote: post #595
ben_w wrote: post #555

Earlier quoted context omitted.

I went from using ChatGPT 3.5 for functions and occasional scripts… … to one of the models in Jan 2024 being able to repeatedly add features to the same single-page web app without corrupting its own work or hallucinating the APIs it had itself previously generated… … to last month using a gifted free week of Claude Code to finish one project and then also have enough tokens left over to start another fresh project w…

So you did some basic web development and built a "not well engineered" greenfield app that you didn't ship, and from that your conclusion is that "writing code is dead"?

In half a week with left-over credit.

What do you think the first half of the credit was spent on?

In addition to the other projects it finished off for me, the reason I say "coding is dead" is that even this mediocre quality code is already shippable. Customers do not give a toss if it has clean code or nicely refactored python backend, that kind of thing is a pain point purely for developers, and when the LLM is the developer then the LLM is the one who gets to be ordered to pay down the technical debt.

The other project (and a third one I might have done on a previous free trial) are as complete as I care to make them. They're "done" in a way I'm not used to being possible with manual coding, because LLMs can finish features faster than I can think of new useful features to add. The limiting factor is my ability to do code review, or would be if I got the more expensive option, as I was on a free trial I could do code review about twice as fast as I burned through tokens (given what others say about the more expensive option that either means I need to learn to code review faster, or my risk tolerance is lower than theirs).

Now, is my new 3-day web app a viable business idea? It would've been shippable as-is 5-6 years ago, I saw worse live around then. Today? Hard to say, if markets were efficient then everyone would know LLMs can create this kind of thing so easily and nobody could charge for them, but people like yourself who disbelieve are an example of markets not being efficient, people like you can have apps like these sold to them.

That said, I try not to look at where the ball is but where it is going. For business ideas, I have to figure out what *doesn't* scale, and do that. Coding *does* scale now, that's why coding is dead.

I expect to return to this project in a month. Have one of the LLMs expand it and develop it for more than 3 the days spent so far, turn it into something I'd actually be happy to sell. Like I said, it seems like we're at "3 weeks" not "3 hours" for a decent MVP by current standards, but the floor is rising fast.

Re: 2025: The Year in LLMs

#608

Post by aspenmartin »

qualifck wrote: post #490
senordevnyc wrote: post #449

Earlier quoted context omitted.

I’ve been reading this comment multiple times a week for the last couple years. Constant assertions that we’re starting to hit limits, plateau, etc. But a cursory glance at where we are today vs a year ago, let alone two years ago, makes it wildly obvious that this is bullshit. The pace of improvement of both models and tooling has been breathtaking. I could give a shit whether you think it’s “exponential”, people li…

People keep saying stuff like this. That the improvements are so obvious and breathtaking and astronomical and then I go check out the frontier LLMs again and they're maybe a tiny bit better than they were last year but I can't actually be sure bcuz it's hard to tell. sometimes it seems like people are just living in another timeline.

You might want to be more specific because benchmarks abound and they paint a pretty consistent picture. LMArena "vibes" paint another picture. I don't know what you are doing to "check" the frontier LLMs but whatever you're doing doesn't seem to match more careful measurement...

You don't actually have to take peoples word for it, read epoch.ai developments, look into the benchmark literature, look at ARC-AGI...

Re: 2025: The Year in LLMs

#609

Post by aspenmartin »

jennyholzer3 wrote: post #578
qualifck wrote: post #490

Earlier quoted context omitted.

People keep saying stuff like this. That the improvements are so obvious and breathtaking and astronomical and then I go check out the frontier LLMs again and they're maybe a tiny bit better than they were last year but I can't actually be sure bcuz it's hard to tell. sometimes it seems like people are just living in another timeline.

"maybe a tiny bit better" is what you say when you've been tricked by snake oil salesman This shit has gotten worse since 2023.

> This shit has gotten worse since 2023.

I would really appreciate it if people could be specific when they say stuff like this because it's so crazy out of line with all measurement efforts. There are an insane amount of serious problems with current LLM / agentic paradigms, but the idea that things have gotten worse since 2023? I mean come on.

Re: 2025: The Year in LLMs

#610

Post by aspenmartin »

llmslave2 wrote: post #168
HPMOR wrote: post #159

Earlier quoted context omitted.

I think this is an open question still and very interesting. Ilya discussed this on the Dwarkesh podcast. But the capabilities of LLMs is clearly exponential and perhaps super exponential. We went from something that could string together incoherent text in 2022 to general models helping people like Terrance Tao and Scott Aaronson write new research papers. LLMs also beat IMO and the ICPC. We have entered the John He…

> But the capabilities of LLMs is clearly exponential and perhaps super exponential By what metric?

- Scaling laws (Chinchilla type)

- METR task horizon

It's a mix, performance gains are bursty but we have been getting a lot of bursts (RLVR, test-time compute, agentic breakthroughs)

Post reply on HN