The difference between the performance of models between 2024 and 2025 has been so stark, that graph really shows it. There are still many people on these forums who seem to think AI’s produce terrible code unless ultra supervised, and I can’t help but suspect some of them tried it a little while ago and just don’t understand how different it is now compared to even quite recently.
2025: The Year in LLMs
271–280 of 643 posts
Re: 2025: The Year in LLMs
#272I'm curious how all of the progress will be seen if it does indeed result in mass unemployment (but not eradication) of professional software engineers.
Even if it will make software engineering drastically more productive, it’s questionable that this will lead to unemployment. Efficiency gains translate to lower prices. Sometimes this leads to very few additional demand, as can be seen with masses of typesetters that lost their jobs. Sometimes this leads to a dramatically higher demand like you can see in the classic Jevons paradox examples of coal and light bulbs. I highly suspect software falls in the latter category
Re: 2025: The Year in LLMs
#273Re: yolo mode
I looked into docker and then realized the problem I'm actually trying to solve was solved in like 1970 with users and permissions.
I just made a agent user limited to its own home folder, and added my user to its group. Then I run Claude code etc as the agent user.
So it can only read write /home/agent, and it cannot read or write my files.
I add myself to agent group so I can read/write the agent files.
I run into permission issues sometimes but, it's pretty smooth for the most part.
Oh also I gave it root to a $3 VPS. It's so nice having a sysadmin! :) That part definitely feels a bit deviant though!
Re: 2025: The Year in LLMs
#274Earlier quoted context omitted.
> exponential progress First you need to define what it means. What's the metric? Otherwise it's very much something you can argue about.
> What's the metric? Language model capability at generating text output. The model progress this year has been a lot of: - “We added multimodal” - “We added a lot of non AI tooling” (ie agents) - “We put more compute into inference” (ie thinking mode) So yes, there is still rapid progress, but these ^ make it clear, at least to me, that next gen models are significantly harder to build. Simultaneously we see a disti…
> Language model capability at generating text output.
That's not a quantifiable sentence. Unless you put it in numbers, anyone can argue exponential/not.
> next gen models are significantly harder to build.
That's not how we judge capability progress though.
> Remind me what was so great about gpt 5? How about gpt4 from from gpt 3?
> Do you even remember the releases?
At gpt 3 level we could generate some reasonable code blocks / tiny features. (An example shown around at the time was "explain what this function does" for a "fib(n)") At gpt 4, we could build features and tiny apps. At gpt 5, you can often one-shot build whole apps from a vague description. The difference between them is massive for coding capabilities. Sorry, but if you can't remember that massive change... why are you making claims about the progress in capabilities?
> Multimodal add ons that no one asked for
Not only does multimodal input training improve the model overall, it's useful for (for example) feeding back screenshots during development.
Re: 2025: The Year in LLMs
#275Earlier quoted context omitted.
That's all very impressive, to be sure. But are you sure you're getting the point? As of 2025, LLMs are now very good at writing new code, creating new imagery, and writing original text. They continue to improve at a remarkable rate. They are helping their users create things that didn't exist before. Additionally, they are now very good at searching and utilizing web resources that didn't exist at training time. So…
> They are helping their users create things that didn't exist before. That is a derived output. That isn't new as in: novel. It may be unique but it is derived from training data. LLMs legitimately cannot think and thus they cannot create in that way.
Yeah you’ve lost me here I’m sorry. In the real world humans work with AI tools to create new things. What you’re saying is the equivalent of “when a human writes a book in English, because they use words and letters that already exist and they already know they aren’t creating anything new”.
Re: 2025: The Year in LLMs
#276Let's talk about the societal cost these models have had on us including their high energy cost and the proliferation of auto-generated slop media used to milk ad revenue, scam people, SEO farm, do propaganda or automate trolling. What about these big corporations collecting an astronomical amount of debt to hoard DRAM and NAND in a way that has crippled the PC market within weeks? And what are they going to do next, put a few dollars in Trump's pocket so that they can rob/loot the US population through bailouts? Who gets to keep all the hardware I wonder?
Nvidia, Samsung, SK Hynix and some other voltures I forgot to mention are making serious bank right now.
Re: 2025: The Year in LLMs
#2772026: The Year of Robots, note it for next year
Re: 2025: The Year in LLMs
#278Re: yolo mode I looked into docker and then realized the problem I'm actually trying to solve was solved in like 1970 with users and permissions. I just made a agent user limited to its own home folder, and added my user to its group. Then I run Claude code etc as the agent user. So it can only read write /home/agent, and it cannot read or write my files. I add myself to agent group so I can read/write the agent file…
I use a qemu vm for running codex cli in yolo mode and use simple ssh based git operations for getting code in and out of there. Works great. And you can also do fun things like let it loose on multiple git projects in one prompt. The vm can run docker as well which helps with containerized tests and other more complicated things. One thing I've started to observe is that you spend more time waiting for tool execution than for model inference. So having a fast local vm is better than a slower remote one.
Re: 2025: The Year in LLMs
#279Earlier quoted context omitted.
Based on quite a few comments recently, it also looks like many have tried LLMs in the past, but haven't seriously revisited either the modern or more expensive models. And I get it. Not everyone wants to keep up to date every month, or burn cash on experiments. But at the same time, people seem to have opinions formed in 2024. (Especially if they talk about just hallucinations and broken code - tell the agent to sea…
Just last week Opus 4.5 decided that the way to fix a test was to change the code so that everything else but the test broke. When people say ”fix stuff” I always wonder if it actually means fix, or just make it look like it works (which is extremely common in software, LLM or not).
Sure, I get an occasional bad result from Opus - then I revert and try again, or ask it for a fix. Even with a couple of restarts, it's going to be faster than me on average. (And that's ignoring the situations where I have to restart myself)
Basically, you're saying it's not perfect. I don't think anyone is claiming otherwise.
Re: 2025: The Year in LLMs
#280Earlier quoted context omitted.
Do you also think LLMs "think"? From what you've described an LLM has not invented anything. LLMs that can reason have a bit more slight of hand but they're not coming up with new ideas outside of the bounds of what a lot of words have encompassed in both fiction and non. Good for you that you've got a fun token of code that's what you've always wanted, I guess. But this type of fantasy take on LLMs seems to be more…
Hang on, you're now saying that if something has ever been described in fiction it doesn't count as invention? So if somebody literally developed a working photon torpedo, that isn't new because "Star Trek Did It"?
Is there any danger an LLM is going to create a working photo torpedo?