[flagged]
Ask HN: Have LLMs Plateaued?
11–20 of 35 posts
Re: Ask HN: Have LLMs Plateaued?
#12[flagged]
Re: Ask HN: Have LLMs Plateaued?
#13AI is beginning to do PHD-level math.
If we're nearing the point where you can spin up 1,000 agents to look for ways improve existing models, then automatically run experiments to validate those ideas, we're more or less at RSI.
As always with AI progress, compute will bottleneck this early on, but a few efficiency improvements could dramatically increase this pace of progress.
I suspect we are at most 24 months from FOOM, but I suspect within about 6-12 months most frontier AI labs will be claiming the majority of their AI research will be AI-driven.
Re: Ask HN: Have LLMs Plateaued?
#14I hope so!
Re: Ask HN: Have LLMs Plateaued?
#15Been using Kimi, and yeah... the American ones seems like heading nowhere new.
Re: Ask HN: Have LLMs Plateaued?
#16We've been seeing terrible performance from Opus 5, so it does look like a plateau. They most certainly reached the marginal returns realm.
Re: Ask HN: Have LLMs Plateaued?
#17I think the big change was agents. The LLM improvements after that point have been relatively minor in impact.
Agents are the proof that LLMs Plateaued : you need the loop to get further.
Re: Ask HN: Have LLMs Plateaued?
#18nowhere near plateau, but right at the inflection point of diminishing returns imo
This is the best take. Return on capital is diminishing from intense competition so improvements get hidden away like the gems they are.
1) Frontier labs have no incentive to give the general public their best anymore; it's instantly distilled off of them. Why not charge governments and big corps real money to use the real good stuff instead? 2) So we get distilled-off-frontier public APIs like 5.6 and Fable/Opus. And the open source labs are distilling off of those. 3) There's a lot of benchmark hacking right now among all the publicly available models, actual usability of Opus for coding is far below its benchmarks suggest. 4) But context window, cybersecurity, logical coherence, and tool usage are absolutely better on Fable and Sol. It looks to me their internal tools definitely even better and not plateauing. But we won't get to use it.
Re: Ask HN: Have LLMs Plateaued?
#19I'm not answering your actual question, but... even if they have totally plateaued technically, there's still some more improvement left to be had from people learning how best to use them (and when not to).
how can you learn to use a tool that keeps changing unpredictably?
Re: Ask HN: Have LLMs Plateaued?
#20In terms of the amount of information in the current LLMs, I think the largest is 5.6 trillion bytes of memory, it is miniscule to what is actually out there. There are over 13 zettabytes of information on the internet. Part of the capacity is about absorbing information. It has a lot further to go. If there are emergent properties with additional knowledge in LLMs, it is scratching the surface. I wonder what will happen with zettabyte computing.