Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2025: The Year in LLMs

simonwillison.net

Post reply on HN
Print view View original

501–510 of 643 posts

Re: 2025: The Year in LLMs

#501

Post by insin »

aspenmartin wrote: post #478
bopbopbop7 wrote: post #471

Earlier quoted context omitted.

Please do provide some data for this "obvious value of coding agents". Because right now the only thing obvious is the increase in vulnerabilities, people claiming they are 10x more productive but aren't shipping anything, and some AI hype bloggers that fail to provide any quantitative proof.

Sure: at my MAANG company, where I watch the data closely on adoption of CC and other internal coding agent tools, most (significant) LOC are written by agents, and most employees have adopted coding agents as WAU, and the adoption rate is positively correlated with seniority. Like a lot of things LLM related (Simon Willison's pelican test, researchers + product leaders implementing AI features) I also heavily "vibe"…

> - Give up when Claude is hopelessly lost

You love to see "Maybe completely waste my time" as part of the normal flow for a productivity tool

Re: 2025: The Year in LLMs

#502

Post by simonw »

_zagj wrote: post #477
simonw wrote: post #189

Earlier quoted context omitted.

If you're not assuming good faith what are you assuming here? What's my motivation? "buttressing commentary on someone else's blog post" That's how link blogs work. I wrote more about my approach to that here: https://simonwillison.net/2024/Dec/22/link-blog/ (And yes, there I go again linking to something I've written from a comment. It's entirely relevant to the point I am making here. That's why I have a blog - so…

> What's my motivation? Are you really going to insult my and others' intelligence like this? Directly or indirectly, your motivation is money. You already offer monthly subscriptions to your blog, and you're clearly trying to build a monetizable brand for yourself as a leading authority on AI, especially as it pertains to software development.

If my motivation was money I would cash in on the reputation I've already built and go and land a Silicon Valley salaried job somewhere.

Sponsorship from my monthly newsletter doesn't come close.

Seriously, do you have any idea how much money I'm leaving on the table right now NOT having a real job in this space?

Being a blogger is wildly financially irresponsible!

Re: 2025: The Year in LLMs

#503

Post by ogou »

andy99 wrote: post #371
weatherlite wrote: post #345

Earlier quoted context omitted.

No idea about training tenserflow models - is it super complex or is it just calling a couple of APIs ? Langchain is literally calling an API. Maybe you need to get good with prompting or whatever, but I don't see where the complexity lies. Please let me know.

Having used both Tensorflow (though I expect they mean PyTorch which is way more popular, and I have also used) and langchain, they are nothing alike. They he ML frameworks are much closer to implementing the mathematics of neural networks, with some abstractions but much closer to the linear algebra level. It requires an understanding of the underlying theory. Langchain is a suite of convenience functions for compos…

You're right, none of these new tools are disciplines. They are vendor specific approaches that are very recent. That's part of my overall point. Who is out there with 2+ years of very narrow tooling experience at another company at a senior level and is available for a rando startup (or desparate enterprise looking for bolt-on AI features) at a fraction of the pay? Not many, I'm sure. We can level up, do training, and maybe stand up a demo project. But that won't satisfy an ATS scan. It's unrealistic.

Re: 2025: The Year in LLMs

#504

Post by ACCount37 »

tim333 wrote: post #383
ACCount37 wrote: post #311

Earlier quoted context omitted.

Honestly, I wouldn't be surprised if a system that's an LLM at its core can attain AGI. With nothing but incremental advances in architecture, scaffolding, training and raw scale. Mostly the training. I put less and less weight on "LLMs are fundamentally flawed" and more and more of it on "you're training them wrong". Too many "fundamental limitations" of LLMs are ones you can move the needle on with better training…

They seem to be missing a bit on learning as you go and thinking about things and getting new insights.

In-context learning and reasoning cover that already, and you could expand on that. Nothing prevents an LLM from fine-tuning itself either, other than its own questionable fine tuning skills and the compute budget.

Re: 2025: The Year in LLMs

#505

Post by windexh8er »

CamperBob2 wrote: post #412
windexh8er wrote: post #370

Earlier quoted context omitted.

> 5 years ago a typical argument against AGI was that computers would never be able to think because "real thinking" involved mastery of language which was something clearly beyond what computers would ever be able to do. Mastery of words is thinking? In that line of argument then computers have been able to think for decades. Humans don't think only in words. Our context, memory and thoughts are processed and occur…

Mastery of words is thinking? That's the crazy thing. Yes, in fact, it turns out that language encodes and embodies reasoning. All you have to do is pile up enough of it in a high-dimensional space, use gradient descent to model its original structure, and add some feedback in the form of RL. At that point, reasoning is just a database problem, which we currently attack with attention. No one had the faintest clue. E…

> ELIZA, ROFL. How'd ELIZA do at the IMO last year?

What's funny is the failure to grasp any contextual framing of ELIZA. When it came out people were impressed by it's reasoning, it's responses. And in your line of defense it could think because it had mastery of words!

But fast forward the current timeline 30 years. You will have been of the same camp that argued on behalf of ELIZA when the rest of the world was asking, confusingly: how did people think ChatGPT could think?

Re: 2025: The Year in LLMs

#506

Post by windexh8er »

CamperBob2 wrote: post #408
windexh8er wrote: post #379

Earlier quoted context omitted.

Wow. You’re using ‘derived’ to imply ‘therefore equivalent.’ That’s a category error. A cookbook is derived from food culture. Does an LLM taste food? Can it think about how good that cookie tastes? A flight simulator is derived from aerodynamics - yet it doesn’t fly. Likewise, text that resembles reasoning isn’t the same thing as a system that has beliefs, intentions, or understanding. Humans do. LLMs don't. Also...…

Ask an LLM what's the difference between a human brain and an LLM. If an LLM could "think" it wouldn't give you the answer it just did. I imagine that sounded more profound when you wrote it than it did just now, when I read it. Can you be a little more specific, with regard to what features you would expect to differ between LLM and human responses to such a question? Right now, LLM system prompts are strongly geare…

> Right now, LLM system prompts are strongly geared towards not claiming that they are humans or simulations of humans. If your point is that a hypothetical "thinking" LLM would claim to be a human, that could certainly be arranged with an appropriate system prompt. You wouldn't know whether you were talking to an LLM or a human -- just as you don't now -- but nothing would be proved either way. That's ultimately why the Turing test is a poor metric.

The mental gymnastics here is entertainment at best. Of course the thinking LLM would give feedback on how it's actually just a pattern model over text - well, we shouldn't believe that! The LLM was trained to lie about it's true capabilities in your own admission?

How about these...

What observable capability would you expect from "true cognitive thought" that a next-token predictor couldn’t fake?

Where are the system’s goals coming from—does it originate them, or only reflect the user/prompt?

How does it know when it’s wrong without an external verifier? If the training data says X and the answer is Y - how will it ever know it was wrong and reach the correct conclusion?

Re: 2025: The Year in LLMs

#507

Post by simonw »

Madmallard wrote: post #472
simonw wrote: post #468

Earlier quoted context omitted.

Weird. I would expect Gemini 3 Pro and Claude Opus 4.5 to run rings around Gemini 1.5 Pro and Claude Sonnet 3.5. How are you running them - regular chat interface or do you have them setup with Claude Code or Gemini CLI?

Using the chat interface primarily with various prompting strategies. I am considering making a thread where I compel others to attempt to get what I'm trying to get out of it and show me their work. The game is only around 25000-30000 LOC in C#.

I'd be happy to join such a thread.

Re: 2025: The Year in LLMs

#508

Post by martinald »

cloudking wrote: post #236
the_mitsuhiko wrote: post #6

> The (only?) year of MCP I like to believe, but MCP is quickly turning into an enterprise thing so I think it will stick around for good.

For connecting agents to third-party systems I prefer CLI tools, less context bloat and faster. You can define the CLI usage in your agent instructions. If the MCP you're using doesn't exist as a CLI, build one with your agent.

Totally agree - wrote this over the holidays which sums it all pretty well https://martinalderson.com/posts/why-im-building-my-own-clis...

Re: 2025: The Year in LLMs

#509

Post by mr_toad »

ako wrote: post #310
crystal_revenge wrote: post #154

Earlier quoted context omitted.

That must have been a long time back. Having lived through the time when web pages were served through CGI and mobile phones only existed in movies, when SVMs where the new hotness in ML and people would write about how weird NNs were, I feel like I've seen a lot more concrete progress in the last few decades than this year. This year honestly feels quite stagnant. LLMs are literally technology that can only reproduc…

> This year honestly feels quite stagnant. LLMs are literally technology that can only reproduce the past. Is this such a big limitation? Most jobs are basically people trained on past knowledge applying it today. No need to generate new knowledge. And a lot of new knowledge is just combining 2 things from the past in a new way.

Most people are capable of long-term learning. Some people are capable of discovering and inventing new things. I think the two are related, and current NN architecture doesn’t allow this. An AI that can cobble together a CRUD application to spec is one thing. An AI that can come up with a new idea for a successful app on its own is a completely different ball game.

Re: 2025: The Year in LLMs

#510

Post by Ianjit »

bopbopbop7 wrote: post #471
aspenmartin wrote: post #467

Earlier quoted context omitted.

The cope + disappointment will be knowing that a large population of HN users will paint a weird alternative reality. There are a multitude of messages about AI that are out there, some are highly detached from reality (on the optimistic and pessimistic side). And then there is the rational middle, professionals who see the obvious value of coding agents in their workflow and use them extensively (or figure out how t…

Please do provide some data for this "obvious value of coding agents". Because right now the only thing obvious is the increase in vulnerabilities, people claiming they are 10x more productive but aren't shipping anything, and some AI hype bloggers that fail to provide any quantitative proof.

The productivity uplift is massive, Meta got a 6-12% productivity uplift from AI coding!

https://youtu.be/1OzxYK2-qsI?si=8Tew5BPhV2LhtOg0

Post reply on HN