Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

41–50 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#41

Post by al_borland »

fnordpiglet wrote: post #21
al_borland wrote: post #2

> Experts in artificial intelligence estimate... The predictions of these "experts" have been drastically wrong for the last 3 years. At what point does someone lose their "expert" title?

It’s hard to take seriously a statement like this that’s grounded in “the last three years” when trends have tended to play out over decades. The fact the goal posts have moved from thousands to hundreds to decades to single digit years for massive transformation should give you a moment of pause in your bag saying. I would however agree there are no experts. Not because of some prediction made on a short time scale…

I said 3 years, because that’s how long ago ChatGPT was released to the public. Since then we’ve been told that all of us should be out of a job in 6 months, every 6 months, for those 3 years.

I agree that change happens on a longer timeline. This is why I’m so tired of statements like this…

“Within a couple of years, possibly much sooner…”

These “experts” are pulling timelines out of the sky, and these predictions are leading to reckless behavior from CEOs and executives which have a material impact on people’s lives. But they get clicks on their blogs and funding for their startup… I guess that’s all that matters.

Re: Humanity isn't ready for the coming intelligence explosion

#42

Post by Retric »

MattRix wrote: post #33
Retric wrote: post #31

Earlier quoted context omitted.

Do you have specific and objective examples of things people got right? From my perspective there’s very slow but very real progress happening in the AI space. I see people making wild predictions in both directions, but in terms of actual unsupervised utility there’s definitely progress abet wildly slower than most hype.

The idea that progress is “slow” in the AI space is absurd. These are some of the fastest growing products and companies of all time. The models are still improving a surprising amount.

I suspected you felt that way even though it hasn’t been my personal experience.

I’ve heard people say older models can’t do X, when I used that way etc. I suspect people are applying their own learning curve as part of their assessment of progress, you get better at writing prompts and it feels like the model improved.

Which is why I’m saying we need some objective metrics to judge predictions of actual capacity.

Re: Humanity isn't ready for the coming intelligence explosion

#43

Post by frognumber »

Retric wrote: post #31
ianm218 wrote: post #16

Earlier quoted context omitted.

Do you have a source on aggregate predictions of experts being drastically wrong? I.e. the AI 2027 guys were memed for being AI lunatics on a lot here and they have been pretty on the money in terms of pace of progress accelerating/ gov moving towards nationalization/ coding agents

Do you have specific and objective examples of things people got right? From my perspective there’s very slow but very real progress happening in the AI space. I see people making wild predictions in both directions, but in terms of actual unsupervised utility there’s definitely progress abet wildly slower than most hype.

This was their prediction for 2026:

"The bet of using AI to speed up AI research is starting to pay off.

OpenBrain continues to deploy the iteratively improving Agent-1 internally for AI R&D. Overall, they are making algorithmic progress 50% faster than they would without AI assistants—and more importantly, faster than their competitors. The AI R&D progress multiplier: what do we mean by 50% faster algorithmic progress?

Several competing publicly released AIs now match or exceed Agent-0, including an open-weights model. OpenBrain responds by releasing Agent-1, which is more capable and reliable.28

People naturally try to compare Agent-1 to humans, but it has a very different skill profile. It knows more facts than any human, knows practically every programming language, and can solve well-specified coding problems extremely quickly. On the other hand, Agent-1 is bad at even simple long-horizon tasks, like beating video games it hasn’t played before. Still, the common workday is eight hours, and a day’s work can usually be separated into smaller chunks; you could think of Agent-1 as a scatterbrained employee who thrives under careful management.29 Savvy people find ways to automate routine parts of their jobs.30

OpenBrain’s executives turn consideration to an implication of automating AI R&D: security has become more important. In early 2025, the worst-case scenario was leaked algorithmic secrets; now, if China steals Agent-1’s weights, they could increase their research speed by nearly 50%.31 OpenBrain’s security level is typical of a fast-growing ~3,000 person tech company, secure only against low-priority attacks from capable cyber groups (RAND’s SL2).32 They are working hard to protect their weights and secrets from insider threats and top cybercrime syndicates (SL3),33 but defense against nation states (SL4&5) is barely on the horizon."

https://ai-2027.com/

That's precisely where we are.

This is eerie. It's like a time traveler. The only delta is Anthropic is in the role of OpenAI.

Re: Humanity isn't ready for the coming intelligence explosion

#44

Post by somenameforme »

g-b-r wrote: post #24

> Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest AI laboratories Or to have AI regulated in their favor

Exactly. Now that Anthropic has gotten to experience that getting the government involved doesn't necessarily mean they get to sit on throne surrounded by a deep moat, I expect their messaging to be adjusted. They're going to want to keep the hype train rolling to keep the dough flowing, but they can't ramp it up to 11 anymore because the government actually is listening now. And Reagan's 9 words have never been more true.

Re: Humanity isn't ready for the coming intelligence explosion

#45

Post by dotdi »

MattRix wrote: post #33
Retric wrote: post #31

Earlier quoted context omitted.

Do you have specific and objective examples of things people got right? From my perspective there’s very slow but very real progress happening in the AI space. I see people making wild predictions in both directions, but in terms of actual unsupervised utility there’s definitely progress abet wildly slower than most hype.

The idea that progress is “slow” in the AI space is absurd. These are some of the fastest growing products and companies of all time. The models are still improving a surprising amount.

It's not that absolute progress is slow, it's extremely slow compared to the predictions. It might be fast in absolute terms, but the "50% of coders will be obsolete by 2023" has been renewed every six months, and it's becoming increasingly clear that there's a real chance it might not ever happen.

Re: Humanity isn't ready for the coming intelligence explosion

#46

Post by frognumber »

al_borland wrote: post #41
fnordpiglet wrote: post #21

Earlier quoted context omitted.

It’s hard to take seriously a statement like this that’s grounded in “the last three years” when trends have tended to play out over decades. The fact the goal posts have moved from thousands to hundreds to decades to single digit years for massive transformation should give you a moment of pause in your bag saying. I would however agree there are no experts. Not because of some prediction made on a short time scale…

I said 3 years, because that’s how long ago ChatGPT was released to the public. Since then we’ve been told that all of us should be out of a job in 6 months, every 6 months, for those 3 years. I agree that change happens on a longer timeline. This is why I’m so tired of statements like this… “Within a couple of years, possibly much sooner…” These “experts” are pulling timelines out of the sky, and these predictions a…

Be a Bayesian, and they'll stop annoying you.

If you have a 5% chance of thermonuclear war each decade for 10 decades, you'll:

- Hear similar annoying statements

- They'll be true

With AI, we don't know if it's one week or one decade. This means we should assign probabilities and consider all possibilities, not get annoyed.

Re: Humanity isn't ready for the coming intelligence explosion

#47

Post by g-b-r »

> Fermi asked why, given the apparent abundance of planets suitable for life, no evidence of other technologically advanced civilisations had been detected. One disquieting possibility is that intelligent life routinely reaches a technological threshold and fails to navigate it,

The Fermi paradox could actually also be taken as an evidence that it's rare (at least) for artificial intelligences to take over a civilization and sprawl and survive for very long times

Re: Humanity isn't ready for the coming intelligence explosion

#48

Post by holmesworcester »

cortesoft wrote: post #27

I couldn't read the whole article, but just from the part I could read: > Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest ai laboratories. I don't know, they also have an incentive to make their technology seem transformative and powerful, and saying that your technology has the power to cause a m…

https://archive.ph/2OWwO

Re: Humanity isn't ready for the coming intelligence explosion

#49

Post by dataviz1000 »

g-b-r wrote: post #32

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes? Or does someone think that they could get magically able to predict big improvements without training?

I built a self-learning recursive agent that finds academic research about using options data to trade, re-creates the research, and then probes and tests for gaps and potential strategies testing against over one year of out-of-sample trading data with one of several strategies that beat SPY by 10x. [0]

One rule is that if a position is opened using the historical data, it can't close the position until the next morning so it isn't a day trading strategy.

I'm curious how this self-learning recursive agent would have preformed in the past 4 months? I don't feel like shelling out $200 to access the data. Do you think that trading strategy will collapse? Whatever the case, if this agent really can perform like that and there isn't a look ahead bias leak in the backtesting (which is definitely a possibility or more likely what happened even though I spent days trying to harden against that), it is game over!

[0] https://github.com/adam-s/alphadidactic

Re: Humanity isn't ready for the coming intelligence explosion

#50

Post by g-b-r »

supertroop wrote: post #37
CamperBob2 wrote: post #5

Humanity doesn't fully leverage the intelligence it already has, so I think people are overestimating the disruption ahead.

Depends what country you are in. The US government seems to be averse to intelligence, whereas China fosters it. Kinda strange that the US is falling behind, I never imagined the day.

> China fosters it

so long as it's not used to dissent from the government

Post reply on HN