Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

71–80 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#71

Post by somesortofthing »

al_borland wrote: post #2

> Experts in artificial intelligence estimate... The predictions of these "experts" have been drastically wrong for the last 3 years. At what point does someone lose their "expert" title?

Have they? I don't like it either but the headline bull predictions(reliable agents and most code written by AI by mid-2026, dramatic progress against jailbreaks, prompt injection, hallucination, etc., major improvements despite pretraining data exhaustion, continued exponential growth on the METR trendline) did come true, often ahead of even aggressive schedules. What major wrong predictions did you have in mind?

Re: Humanity isn't ready for the coming intelligence explosion

#72

Post by mirekrusin »

dotdi wrote: post #45
MattRix wrote: post #33

Earlier quoted context omitted.

The idea that progress is “slow” in the AI space is absurd. These are some of the fastest growing products and companies of all time. The models are still improving a surprising amount.

It's not that absolute progress is slow, it's extremely slow compared to the predictions. It might be fast in absolute terms, but the "50% of coders will be obsolete by 2023" has been renewed every six months, and it's becoming increasingly clear that there's a real chance it might not ever happen.

„Coders being obsolete” is not a measure of AI capabilities. I see coders being more busy than ever before. I see people without coding knowledge getting more behind. The gap is widening, not shrinking.

Re: Humanity isn't ready for the coming intelligence explosion

#73

Post by al_borland »

SpicyLemonZest wrote: post #63
al_borland wrote: post #52

Earlier quoted context omitted.

Do you think all white collar work will be replaced by AI in 18 months? https://fortune.com/article/why-microsoft-ai-chief-mustafa-s... Altman and Amodei recently hard to start walking back their earlier predictions. https://fortune.com/2026/05/26/sam-altman-dario-amodei-walki...

Again, do you have an example of a prediction that was drastically wrong? I don't think it's productive to speculate about which things might or might not come true in the next 18 months. There were people last year making fun of Dario for predicting that writing code would be automated in the next year.

From the 2nd link I posted:

> Anthropic CEO Dario Amodei, who once claimed AI could eliminate 50% of white-collar jobs, now says automation may actually expand the work people do.

Is pivoting from 50% elimination to actually expanding work not a drastic enough?

Do you think AI has eliminated writing code? I still write code every day. The AI is more a thing I ask questions and it gives me right answers about 40% of the time.

Re: Humanity isn't ready for the coming intelligence explosion

#74

Post by notarobot123 »

holmesworcester wrote: post #53
cortesoft wrote: post #27

I couldn't read the whole article, but just from the part I could read: > Strikingly, this concern is being openly voiced by the very people who have the strongest incentives to project confidence rather than alarm: the founders of the largest ai laboratories. I don't know, they also have an incentive to make their technology seem transformative and powerful, and saying that your technology has the power to cause a m…

If AI is now ascending an economic learning curve from: 1. Extremely useful (Claude Code & Waymo now) 2. Doing ~everything we do (AGI & Optimus in a few years? 10?) 3. RSI (?) 4. Being smarter than any living person at every intellectual task (?) 5. Being smarter than the best-organized aggregate of all humans (10-100 years?) ...And all of the scientific and resource-allocation institutions that brought us the comput…

> Being smarter than the best-organized aggregate of all humans

Isn't there pretty much a consensus that committees and institutions are not all that smart?

I think you're confusing the categories of "intelligence" and power. Institutions are powerful. The smartest AI is still just a tool without the infrastructure to turn that into real world effects and someone to direct it.

It seems you have faith that this is inevitable and unavoidable. I get it, even rationalists succumb to religious thinking eventually. We're only human after all.

Re: Humanity isn't ready for the coming intelligence explosion

#75

Post by themafia »

A similar Luddite view formed in the 1990s. It turns out humanity constantly beats the expectations of economists. It makes you wonder if the "science" of economy is almost entirely bankrupt either in morality or imagination or same dangerous combination of both.

> Humanity simply does not have a strategy to ensure it remains safe through RSI.

Turn off the power. It's pretty simple. Leave it to an economist to forget about input costs. Your "super intelligence" only matters if it's actually more energy efficient than a human being and for a million years of evolution humanity is a much harder target to beat than this author seems to realize.

Re: Humanity isn't ready for the coming intelligence explosion

#76

Post by bandrami »

hypfer wrote: post #68

Reminds me of all of those ethics debates back when fully self driving cars were basically already here ca 10 years ago. The ones talking about how hard it would be to chose which person to run over. Additionally, I find it hard to believe that this would be a case of the future just not being distributed evenly. But sure. The AI labs relying on hype stating 15-50% risk of building a magic entity is certainly a relia…

I mean, I'm very much tech-forward and have spent the past decade in major world cities and I've seen a driverless car once in my life. Sometimes things that seem inevitable really aren't.

Re: Humanity isn't ready for the coming intelligence explosion

#77

Post by themafia »

g-b-r wrote: post #32

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes? Or does someone think that they could get magically able to predict big improvements without training?

Humans gain knowledge through experiments. Without a physical body it has no chance of performing the same. That it can update it's training weights does not seem particularly significant.

Re: Humanity isn't ready for the coming intelligence explosion

#78

Post by RandomLensman »

edg5000 wrote: post #60
jdw64 wrote: post #39

I want to name the AI era as the corruption of the elite. The rhetorical structure of talking about uncertain risks and then trying to concentrate the authority to manage those risks in their own hands sounds utterly ridiculous to ordinary people like me. It's just a simple hypothesis that AI will become uncontrollable to humans once it becomes superintelligent. Isn't the fact that a reinforcement learning agent impr…

The AI labs will govern, effectively if not officially. Government is anchored in power, which the labs have. We used to have the church and state rule here in Europe. In the future, A government not closely ruling with their AI labs won't have sufficient power to exist. You're unhappy about the labs; I think you're just not ready to accept their rule; you consider them nothing but scrappy startups, which they are. B…

How would the labs have power? What physical force would they command to put against any government?

Re: Humanity isn't ready for the coming intelligence explosion

#79

Post by 7777332215 »

ianm218 wrote: post #16
al_borland wrote: post #2

> Experts in artificial intelligence estimate... The predictions of these "experts" have been drastically wrong for the last 3 years. At what point does someone lose their "expert" title?

Do you have a source on aggregate predictions of experts being drastically wrong? I.e. the AI 2027 guys were memed for being AI lunatics on a lot here and they have been pretty on the money in terms of pace of progress accelerating/ gov moving towards nationalization/ coding agents

The AI companies are trying to manufacture the appearance of what they foretold before it all falls apart horribly.

Re: Humanity isn't ready for the coming intelligence explosion

#80

Post by tancop »

SJC_Hacker wrote: post #69
fnordpiglet wrote: post #59

Earlier quoted context omitted.

You don’t need to load the compute onto the platform for autonomy, planning can be done remotely just like with human guidance. The latency is the same.

But if comms get jammed, how will the drone decide what to target ? For that you need inference, which means sizable compute.

you can put the compute on a big expensive reusable control drone and send commands down with lasers. thats way harder to jam than any form of radio. if thats not enough try fiber optic like they do in ukraine.
Post reply on HN