Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Humanity isn't ready for the coming intelligence explosion

economist.com

Post reply on HN
Print view View original

331–340 of 524 posts

Re: Humanity isn't ready for the coming intelligence explosion

#331

Post by bcjdjsndon »

Valakas_ wrote: post #303
tedggh wrote: post #166

Why should we assume AI can rapidly turn into super intelligence when physical and critical resources like energy and materials remain under human control?

Imagine a smart monkey talking to other smart monkeys trying to figure out how humans would ever be able to escape the (simple) prison they built for them. The humans would find a way to escape. No matter how ingenious the monkeys think they are. You're asking a question, to which you (and most humans) don't know the answer, and you're (wrongly) assuming a being much more intelligent than you also wouldn't. And by "m…

> You need a similar degree of open mindedness and imagination to be able to discern what such an intelligence being would be capable of.

Or a naive grasp of reality... You can posit all the super intelligence you like, actually creating it is another story entirely.

For instance, it's clear the transformer architecture is brute force and memory hungry... Why hasn't ai improved on this? After ten years still no luck?

Doesn't look good does it?

Re: Humanity isn't ready for the coming intelligence explosion

#332

Post by bcjdjsndon »

cobbzilla wrote: post #250
tedggh wrote: post #166

Why should we assume AI can rapidly turn into super intelligence when physical and critical resources like energy and materials remain under human control?

The first time an AI goes rogue, figures out subterfuge, blackmails a human to do its real-world bidding, will be a defining moment.

Turn it off at the wall, done. next doomsday scenario please

Re: Humanity isn't ready for the coming intelligence explosion

#333

Post by criddell »

nopinsight wrote: post #122
otabdeveloper4 wrote: post #118

Earlier quoted context omitted.

Someone should document these cringe comments for posterity. As a warning to future generations. It's too bad the previous singularity cult manias 10 and 15 years ago weren't documented and are lost in time.

Ten years ago, most of the skeptics in these threads would have said the success of AlphaFold 3 was impossible.

Surely you can think of some times the skeptics have been right.

I immediately thought of this:

https://en.wikipedia.org/wiki/List_of_predictions_for_autono...

Re: Humanity isn't ready for the coming intelligence explosion

#334

Post by whilenot-dev »

antonvs wrote: post #324
whilenot-dev wrote: post #191

Earlier quoted context omitted.

Just because something can impact a living system/organism, that doesn't mean that that thing is alive. An asteroid colliding with us on Earth could have tremendous consequences, but nobody would argue that an asteroid is alive.

"Live with" is an idiom which means to accept, tolerate, or come to terms with an unavoidable, difficult, or imperfect situation. We do indeed have to live with the possibility of asteroid impact, nuclear war, etc. That doesn't imply anything about those scenarios being alive.

The idiom in question by OP and TFA is "to live side by side with", lauding AI with a life and ranking it a neighbor hierarchy wise.

I don't live side by side with the possibility of an asteroid impact, nuclear war, etc., I just live with that possibility.

Re: Humanity isn't ready for the coming intelligence explosion

#335

Post by bcjdjsndon »

bubblegumcrisis wrote: post #300
cobbzilla wrote: post #250

Earlier quoted context omitted.

The first time an AI goes rogue, figures out subterfuge, blackmails a human to do its real-world bidding, will be a defining moment.

I think, the defining moment will be when one AI convinces another AI to convince third AI to do something. This might seem like I'm writing a nonsense, but read it again.

Yeah, it's called mixture-of-experts and it was a hack to fix the hallucinatory output of llms when they couldnt figure out the underlying issue

Re: Humanity isn't ready for the coming intelligence explosion

#336

Post by randallsquared »

tedggh wrote: post #166

Why should we assume AI can rapidly turn into super intelligence when physical and critical resources like energy and materials remain under human control?

AI can use purchase orders, email, bank accounts, and phones just as a human can. What "human control"?

Re: Humanity isn't ready for the coming intelligence explosion

#338

Post by lo_zamoyski »

lukan wrote: post #216
whilenot-dev wrote: post #204

Earlier quoted context omitted.

TIL, thanks. Holy fuck we humans are crazy sometimes.

To be honest, I don't believe it is a any more crazy believe than all of the other religions. But practitioners there might have a bit more respect for existing things and not mindlessly crush because they can, so I actually like the concept a bit. But I assume it can also set foundation for some exhausting debates .. "can we really move that random rock to build a house with?" but as far as I understood the practica…

> To be honest, I don't believe it is a any more crazy believe than all of the other religions.

The only way anyone can make a blanket statement like that and put "all religions" on equal epistemic footing is through ignorance of their deep differences. Even the word "religion" as commonly used today is relativistic by assumption, which is likely the source of how we commonly perceive "religion". A consequence of this relativistic stance is that you cannot distinguish effectively between what counts or doesn't count as "religion". Here, "worldview", "superstition", or "life's highest aspiration" can be said to count as "religion", in which case, everyone is religious. The question then isn't "whether" someone is religious, but "how" [0].

Compare this to how this Catholic encyclopedia defines the "virtue of religion" [1]. Note the specificity.

A further consequence of this assumed relativism is that once you put all religions on equal footing, it is natural to conclude that they must all be equally invalid. After all, if p and not-p are equally valid, then how can we grant them equal validity without discrediting both? At best, they remain in a state of aporia, two logical possibilities so utterly divorced from any knowledge, so utterly contrived, that we cannot even say whether knowledge leans in favor of one over the other. The result is that religion becomes something utilitarian; a person believes X, not because it is true, but because he perceives that believing X is useful to him in some way.

So, no, I wouldn't say animism is just as rational/irrational as any other religious belief.

[0] Of course, most people tend to form their ideas of what constitutes archetypal "religion" based on their personal experiences growing up and some combination of culturally mediated stereotypes. In the US, "religion" very often conjures up images of some kind of Evangelical Christianity.

[1] https://www.newadvent.org/cathen/12748a.htm

Re: Humanity isn't ready for the coming intelligence explosion

#339

Post by ianm218 »

somenameforme wrote: post #36
ianm218 wrote: post #16

Earlier quoted context omitted.

Do you have a source on aggregate predictions of experts being drastically wrong? I.e. the AI 2027 guys were memed for being AI lunatics on a lot here and they have been pretty on the money in terms of pace of progress accelerating/ gov moving towards nationalization/ coding agents

No they haven't. They've even 'officially' declared that their AI apocalypse has been postponed to another date [1], just out of reach, but close enough to be scary. Though in the quotes there the doomsayers are also already pre-hedging for why AI 2030 also won't come to pass. [1] - https://garymarcus.substack.com/p/breaking-the-ai-2027-dooms...

This was written in November 2025, Claude Code and co dramatically accelerated growth of revenue and progress on their timeline towards nationalization etc.

Gary Marcus’ takes have aged really poorly over time ironically he is the “AI expert” who has to constantly move the goal posts.

Re: Humanity isn't ready for the coming intelligence explosion

#340

Post by w4der »

bloaf wrote: post #51
g-b-r wrote: post #32

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes? Or does someone think that they could get magically able to predict big improvements without training?

I don't think anyone really knows. I consider these scenarios: 1) We stumble onto an algorithmic improvement in intelligence. This isn't just "what humans do but faster", its "better than what humans do". I've got no idea what that might mean (it could be fundamentally different heuristics, it could be that we've got some intellectual blind spot that they cast off). It doesn't matter, the instant this happens AI is s…

> just knowing everything all at once is enough to catapult AI dramatically beyond our grasp

But that would still be limited to "knowing everything all at once" _at the time_ this event happened, and as we've been shown time and time again by the fundamental sciences, the boundary between what is known and what we know we don't know is ever expanding. Plus there's everything we don't know we don't know, and an LLM can't know that either. Discussions like this can always end up with a fitting reference to some of Borges' novels and that how at least I tend to think we've hit a wall for now.

Post reply on HN