Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2025: The Year in LLMs

simonwillison.net

Post reply on HN
Print view View original

201–210 of 643 posts

Re: 2025: The Year in LLMs

#201

Post by Razengan »

My experience with AI so far: It's still far from "butler" level assistance for anything beyond simple tasks.

I posted about my failures to try to get them to review my bank statements [0] and generally got gaslit about how I was doing it wrong, that I if trust them to give them full access to my disk and terminal, they could do it better.

But I mean, at that point, it's still more "manual intelligence" than just telling someone what I want. A human could easily understand it, but AI still takes a lot of wrangling and you still need to think from the "AI's PoV" to get the good results.

[0] https://news.ycombinator.com/item?id=46374935

----

But enough whining. I want AI to get better so I can be lazier. After trying them for a while, one feature that I think all natural-language As need to have, would be the ability to mark certain sentences as "Do what I say" (aka Monkey's Paw) and "Do what I mean", like how you wrap phrases in quotes on Google etc to indicate a verbatim search.

So for example I could say "[[I was in Japan from the 5th to 10th]], identify foreign currency transactions on my statement with "POS" etc in the description" then the part in the [[]] (or whatever other marker) would be literal, exactly as written, but the rest of the text would be up to the AI's interpretation/inference so it would also search for ATM withdrawals etc.

Ideally, eventually we should be able to have multiple different AI "personas" akin to different members of household staff: your "chef" would know about your dietary preferences, your "maid" would operate your Roomba, take care of your laundry, your "accountant" would do accounty stuff.. and each of them would only learn about that specific domain of your life: the chef would pick up the times when you get hungry, but it won't know about your finances, and so on. The current "Projects" paradigm is not quite that yet.

Re: 2025: The Year in LLMs

#202

Post by Madmallard »

aoeusnth1 wrote: post #157
llmslave2 wrote: post #144

Earlier quoted context omitted.

Pretty neat how this exponential progress hasn't resulted in exponential productivity. Perhaps you could explain your perspective on that?

It has! CLs/engineer increased by 10% this year. LLMs from late 2024 were nearly worthless as coding agents, so given they have quadrupled in capability since then (exponential growth, btw), it's not surprising to see a modestly positive impact on SWE work. Also, I'm noticing you're not explaining yourself :)

LLMs a year ago were more able to do a complex project I've repeatedly tried to do than they are now.

Re: 2025: The Year in LLMs

#203

Post by Madmallard »

syndacks wrote: post #89

I can’t get over the range of sentiment on LLMs. HN leans snake oil, X leans “we’re all cooked” —- can it possibly be both? How do other folks make sense of this? I’m not asking for a side, rather understanding the range. Does the range lead you to believe X over Y?

I use them daily and I actively lose progress on complex problems and save time on simple problems.

Re: 2025: The Year in LLMs

#204

Post by lukaslalinsky »

Madmallard wrote: post #200
didip wrote: post #67

Indeed. I don't understand why Hacker News is so dismissive about the coming of LLMs, maybe HN readers are going through 5 stages of grief? But LLM is certainly a game changer, I can see it delivering impact bigger than the internet itself. Both require a lot of investments.

Have you tried using it for anything actually complicated? Lol. It's worse than nothing at all.

I think the split between vibe coding and AI-assisted coding will only widen over time. If you ask LLMs to do something complex, they will fail and you waste your time. If you work with them as a peer, and you delegate tasks to them, they will succeed and you save your time.

Re: 2025: The Year in LLMs

#205

Post by Madmallard »

websiteapi wrote: post #24

I'm curious how all of the progress will be seen if it does indeed result in mass unemployment (but not eradication) of professional software engineers.

Why would it?

The ability to accurately describe what you want with all constraints managed and with proactive design is the actual skill. Not programming. The day PMs can do that and have LLMs that can code to that, is the day software engineers en masse will disappear. But that day is likely never.

The non-technical people I've ever worked for were hopelessly terrible at attention to detail. They're hiring me primarily for that anyway.

Re: 2025: The Year in LLMs

#206

Post by cube00 »

andrewinardeer wrote: post #108

Thank you. Enjoyed this read. AI slop videos will no doubt get longer and "more realistic" in 2026. I really hope social media companies plaster a prominent banner over them which screams, "Likely/Made by AI" and give us the option to automatically mute these videos from our timeline. That would be the responsible thing to do. But I can't see Alphabet doing that on YT, xAI doing that on X or Meta doing that on FB/Ins…

> social media companies plaster a prominent banner over them

Not going to happen as the social media companies realise they can sell you the AI tools used to post slop back onto the platform.

Re: 2025: The Year in LLMs

#207

Post by PeterHolzwarth »

syndacks wrote: post #89

I can’t get over the range of sentiment on LLMs. HN leans snake oil, X leans “we’re all cooked” —- can it possibly be both? How do other folks make sense of this? I’m not asking for a side, rather understanding the range. Does the range lead you to believe X over Y?

I think it may be all summed up by Roy Amara's observation that "We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run."

Re: 2025: The Year in LLMs

#208

Post by weatherlite »

ogou wrote: post #97

This is a good tooling survey of the past year. I have been watching it as a developer re-entering the job market. The job descriptions closely parallel the timeline used in the post. That's bizarre to me because these approaches are changing so fast. I see jobs for "Skill and Langchain experts with production-grade 0>1 experience. Former founders preferred". That is an expertise that is just a few months old and sta…

> Skill and Langchain experts with production-grade 0>1 experience.

Also , it's just normal backend work - calling a bunch of APIs. What am I missing here?

Re: 2025: The Year in LLMs

#209

Post by waldrews »

crystal_revenge wrote: post #154
waldrews wrote: post #3

Remember, back in the day, when a year of progress was like, oh, they voted to add some syntactic sugar to Java...

That must have been a long time back. Having lived through the time when web pages were served through CGI and mobile phones only existed in movies, when SVMs where the new hotness in ML and people would write about how weird NNs were, I feel like I've seen a lot more concrete progress in the last few decades than this year. This year honestly feels quite stagnant. LLMs are literally technology that can only reproduc…

I'm being hyperbolic of course, but I'm a little dismissive of the progress that happened since the days of BBS's and car based cell phones - we just got more connectivity, more capacity, more content, bigger/faster. Likewise, my attitude toward machine learning before 2023 is a smug 'heh, these computer scientists are doing undisciplined statistics at scale, how nice for them.' Then all of a sudden the machines woke up and started arguing with me, coherently, even about niche topics I have a PhD in. I can appreciate in retrospect how much of the machine learning progress ultimately went into that, but, like fusion, the magic payoff was supposed to be decades away and always remain decades away. This wasn't supposed to happen in my lifetime. 2025 progress isn't the 2023 shock, but this was the year LLM's-as-programmers (and LLM's-as-mathematicians, and...) went from 'isn't that cute, the machine is trying' to 'an expert with enough time would make better choices than the machine did,' and that makes for a different world. More so than, going from a Commodore Vic 20 with 4k of RAM and a modem to the latest Macbook.

Re: 2025: The Year in LLMs

#210

Post by Madmallard »

dang wrote: post #94
skydhash wrote: post #11

[flagged]

Could you please stop posting dismissive, curmudgeonly comments? It's not what this site is for, and destroys what it is for. We want curious conversation here. https://news.ycombinator.com/newsguidelines.html

His comment is far better than the rampant astroturfing from stakeholders going on everywhere on this website that is being mitigated not at all whatsoever. There is a wealth of information present suggesting these things are so bad for everyone in so many ways.
Post reply on HN