Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Failing to Understand the Exponential, Again

julian.ac

Post reply on HN
Print view View original

181–190 of 266 posts

Re: Failing to Understand the Exponential, Again

#181

Post by peab »

crazygringo wrote: post #7

> Given consistent trends of exponential performance improvements over many years and across many industries, it would be extremely surprising if these improvements suddenly stopped. I'm sure people were saying that about commercial airline speeds in the 1970's too. But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors. With LLM's at the moment, the limiting fac…

There's a Mulla Nasrudin joke that's sort of relevant here:

Nasrudin is on a flight, when suddenly the pilot comes on the intercom, saying, "Passengers, we apologize, but we have experienced an engine burn-out. The plane can still fly on the remaining three engines, but we'll be delayed in our arrival by two hours."

Nasrudin speaks up "let's not worry, what's 2 hours really"

A few minutes later, the airplane shakes, and passengers see smoke coming out of another engine. Again, the intercom crackles to life.

"This is your captain speaking. Apologies, but due to a second engine burn-out, we'll be delayed by another two hours."

The passengers are agitated, but the Mulla once again tries to remains calm.

Suddenly, the third engine catches fire. Again, the pilot comes on the intercom and says, "I know you're all scared, but this is a very advanced aircraft, and it can safely fly on only a single engine. But we will be delayed by yet another two hours."

At this, Nasrudin shouts, "This is ridiculous! If one more engine goes, we'll be stuck up here all day"

Re: Failing to Understand the Exponential, Again

#182

Post by jijijijij »

Nevermark wrote: post #167
jijijijij wrote: post #162

Earlier quoted context omitted.

That's fallacious reasoning, you are extrapolating from survivorship bias. A lot of technologies, genes, or species have failed along the way. You are also subjectively attributing progression as improvements, which is problematic as well, if you speak about general trends. Evolution selects for adaptation not innovation. We use the theory of evolution to explain the emergence of complexity, but that's not the sole d…

See replies to sibling comments.

Doesn't answer the core fallacy. Historical "technological progress" can't be used as argument for any particular technology. Right now, if we are talking about AI, we're talking about specific technologies, which may just as well fail and remain inconsequential in the grand scheme of things, like most technologies, most things really, did in the past. Even more so since we don't understand much anything in either human or artificial cognition. Again and again, we've been wrong about predicting the limits and challenges in computation.

You see, your argument is just bad. You are merely guessing like everyone else.

Re: Failing to Understand the Exponential, Again

#183

Post by Earw0rm »

analog31 wrote: post #87
Earw0rm wrote: post #70

Earlier quoted context omitted.

I'm curious as to whether the consensus is that the observed behaviour of COVID waves was ever fully and satisfactorily explained - the tend to grow exponentially but then seemingly saturate at a much lower point than a naïve look at the curve might suggest?

It would probably be hard to do. The really huge factor may be easier to study, since we know where and when every vaccine dose was administered. The behavioral factors are likely to be harder to measure, and would have been masked by the larger effect of vaccination. We don't really know the extent of social isolation over geography, demographics, time, etc..

There's human behavioural factors yes, but I was kinda wondering about the virus itself, the R number seemed to fluctuate quite a bit, with waves peaking fast and early and then receding equally quickly.. I know there were some ideas around asymptomatic spread and superspreaders (both people with highly connected social graphs, and people shedding far more active virus than the median), I just wondered whether anyone had built a model that was considered to have accurately reproduced the observed behaviour of number of positive tests and symptomatic cases, and the way waves would seemingly saturate after infecting a few % of the population.

Re: Failing to Understand the Exponential, Again

#184

Post by hnlmorg »

tim333 wrote: post #159
hnlmorg wrote: post #137

Earlier quoted context omitted.

My point is that the limits of LLMs will be hit long before we they start to take on human capabilities. The problem isn’t that exponential growth is hard to visualise. The problem is that LLMs, as advanced and useful a technique as it is, isn’t suited for AGI and thus will never get us even remotely to the stage of AGI. The human like capabilities are really just smoke and mirrors. It’s like when people anthropomorp…

There is no particular reason why AI has to stick to language models though. Indeed if you want human like thinking you pretty much have to go beyond language as we do other stuff too if you see what I mean. A recent example: "Google DeepMind unveils its first “thinking” robotics AI" https://arstechnica.com/google/2025/09/google-deepmind-unvei...

> There is no particular reason why AI has to stick to language models though.

There’s no reason at all. But that’s not the technology that’s in the consumer space, growing exponentially, gaining all the current hype.

So at this point in time, it’s just a theoretical future that will happen inevitably but we don’t know when. It could be next year. It could be 10 years. It could be 100 years or more.

My prediction is that current AI tech plateaus long before any AGI-capable technology emerges.

Re: Failing to Understand the Exponential, Again

#185

Post by Voultapher »

danlitt wrote: post #156
jsnell wrote: post #148

Earlier quoted context omitted.

> but then why are we comparing such distinct and unrelated tasks as ... Because a few years ago the LLMs could only do trivial tasks that a child could do, and now they're able to do complex research and software development tasks. If you just have the trivial tasks, the benchmark is saturated within a year. If you just have the very complex tasks, the benchmark is has no sensitivity at all for years (just everythin…

The key word is "unrelated"! Being able to count the number of words in a paragraph and being able to train an image classifier are so different as to be unrelated for all practical purposes. The assumption underlying this kind of a "benchmark" is that all tasks have a certain attribute called complexity which is a numerical value we can use to discriminate tasks, presumably so that if you can complete tasks up to a…

Well put, the metric is cherry picked to further the narrative.

Re: Failing to Understand the Exponential, Again

#186

Post by vonnik »

To the people who claim that we’re running out of data, I would just say: the world is largely undigitized. The Internet digitized a bunch of words but not even a tiny fraction of all that humans express every day. Same goes for sound in general. CCTV captures a lot of images, far more than social media, but it is poorly processed and also just a fraction of the photons bouncing off objects on earth. The data part of this equation has room to grow.

Re: Failing to Understand the Exponential, Again

#187

Post by bawolff »

Inufu wrote: post #140
bawolff wrote: post #134

Earlier quoted context omitted.

That feels like you're moving the goal posts a bit. Exponential growth over the short term is very uninteresting. Exponential growth is exciting when it can compound. E.g. if i offered you an investing opportunity 500% / per year compounded daily - that's amazing. If the fine print is that that rate will only last for the very near term (say a week), then it would be worse than a savings account.

Well, growth has been on this exponential already for 5+ years (for the METR eval), and we are at the point where models are very close to matching human expert capabilities in many domains - only one or two more years of growth would put us well beyond that point. Personally I think we'll see way more growth than that, but to see profound impacts on our economy you only need to believe the much more conservative ass…

> we are at the point where models are very close to matching human expert capabilities in many domains

That's a bold claim. I don't think it matches most people's experiences.

If that was really true people wouldn't be talking about exponential growth. You don't need exponential growth if you are already almost at your destination.

Re: Failing to Understand the Exponential, Again

#188

Post by Yoric »

jsnell wrote: post #43
crazygringo wrote: post #7

> Given consistent trends of exponential performance improvements over many years and across many industries, it would be extremely surprising if these improvements suddenly stopped. I'm sure people were saying that about commercial airline speeds in the 1970's too. But a lot of technologies turn out to be S-shaped, not purely exponential, because there are limiting factors. With LLM's at the moment, the limiting fac…

Indeed you can't be sure. But on the other hand a bunch of the commentariat has been claiming (with no evidence) that we're at the midpoint of the sigmoid for the last three years. They were wrong. And then you had the AI frontier lab insiders who predicted an accelerating pace of progress for the last three years. They were right. Now, the frontier labs rarely (never?) provide evidence either, but they do have about…

There are a few other limitations, in particular how much energy, hardware and funding we (as a society) can afford to throw at the problem, as well as the societal impact.

AI development is currently given a free pass on these points, but it's very unclear how long that will last. Regardless of scientific and technological potential, I believe that we'll hit some form of limit soon.

Re: Failing to Understand the Exponential, Again

#189

Post by senordevnyc »

coldtea wrote: post #150
senordevnyc wrote: post #117

Earlier quoted context omitted.

Except it’s not been five years, it’s been at most three, since approximately no one was using LLMs prior to ChatGPT’s release, which was just under three years ago. We did have Copilot a year before that, but it was quite rudimentary. And really, we’ve had even less than that. The first large scale reasoning model was o1, which was released 12 months ago. More useful coding agents are even newer than that. This narr…

> Except it’s not been five years, it’s been at most three, Why would it be "at most" 3? We had Chat GPT commercially available as private beta API on 2020. It's only the mass public that got 3.5 3 years ago. But those who'd do the noticing as per my argument is not just Joe Public (which could be oblivious), but people already starting in 2020, and includes people working in the space, who worked with LLM and LLM-li…

No, we didn’t. We had the GPT-3 API available in 2020, and approximately no one was using it.

Re: Failing to Understand the Exponential, Again

#190

Post by poopiokaka »

lairv wrote: post #132
poopiokaka wrote: post #92

“Guy who personally benefits from AI hype says we aren’t in a bubble” - don’t we have enough of these already ??

(I'm the one who posted the url, not author post) Julian Schrittwieser (author of this post) has been in AI for a long time, he was in the core team who worked on AlphaGo, AlphaZero and MuZero at DeepMind, you can see him in the AlphaGo movie. While it doesn't make his opinion automatically true, I think it makes it worth considering, especially since he's a technical person, not a CEO trying to raise money "extrapol…

Benchmarks are notoriously easy to fake. Also he doesn’t need to be a CEO trying to raise money in order to have an incentive here to push this agenda / narrative. He has a huge stock grant from Anthropic that will go to $0 when the bubble pops
Post reply on HN