Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI doomism is quickly becoming indistinguishable from an apocalyptic religion

twitter.com

Post reply on HN
Print view View original

41–50 of 244 posts

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#41

Post by f_devd »

gerikson wrote: post #12
drcode wrote: post #7

A religion is arguably when you refuse to engage with other people's arguments Lecun has refused to engage with anyone's arguments, so why are the people worried about AI ex-risk the religion?

> A religion is arguably when you refuse to engage with other people's arguments I've never heard of that definition.

Neither have I but it actually lines up pretty well depending on your read of GP. Large religions often use a proving-a-negative with a pascals wager to achieve an inarguable story regardless of content. And cult religions generally do this with an Us-vs-Them approach which usually prevents members of believing external arguments.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#42

Post by fatherzine »

survirtual wrote: post #29
moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

AI can also completely liberate all of us. This fear mongering is irrational. We already live in an extremely advanced reality, with organic life having organic nanomachines performing incredibly complex operations in a universe, in realtime, performing impossibly complex calculations. Even to simulate a square meter of our universe in realtime is outlandishly beyond our compute access. Perhaps AI unleashes the creat…

"liberation" is a false idol.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#43

Post by version_five »

lumenwrites wrote: post #20
PaulHoule wrote: post #6

There is "A.I. doomism" in general and there is also a specific cult-like group at the lesswrong website that really is an apocalyptic religion. It is astonishing how anything that deviates from its dogma in any way has to have "trigger warnings" and the senseless comments that users write on anything that suggests there could be intrinsic limits on intelligence. There ideology is such that a higher intelligence is g…

This is an inaccurate and ignorant comment that does not engage with the arguments people on LessWrong are making. If you have something smart to say about why these people are wrong in their beliefs, please do. Name calling a group of people a "cult" or "apocalyptic religion" is not an argument that contributes anything interesting to the conversation. If anyone is actually curious about what these people are saying…

  here's a list of their arguments for why AI is extremely dangerous and extremely likely to kill us all
Not familiar with them in the least, but the fact that they're a group that's assembled such a list basically proves they're something of a doomsday cult, no?

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#44

Post by blibble »

moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

> When my best bet for humanity is a global nuclear war resetting civilization to give us a chance to deal with AGI issue down the line - I'm fine with religious doomism.

I'd settle for corporate death sentence for any companies developing it with life imprisonment for any employees/corporate officers involved at any level whatsoever (or VCs funding it)

including selling chips/compute to companies involved

any potential benefits from AI are not worth even a 10% risk of it wiping out humanity

this position is going to become more common as the media (and electorates) grasp what these companies are attempting to do

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#45

Post by cwmoore »

lumenwrites wrote: post #20
PaulHoule wrote: post #6

There is "A.I. doomism" in general and there is also a specific cult-like group at the lesswrong website that really is an apocalyptic religion. It is astonishing how anything that deviates from its dogma in any way has to have "trigger warnings" and the senseless comments that users write on anything that suggests there could be intrinsic limits on intelligence. There ideology is such that a higher intelligence is g…

This is an inaccurate and ignorant comment that does not engage with the arguments people on LessWrong are making. If you have something smart to say about why these people are wrong in their beliefs, please do. Name calling a group of people a "cult" or "apocalyptic religion" is not an argument that contributes anything interesting to the conversation. If anyone is actually curious about what these people are saying…

That link is not at all a list of "why AI is extremely dangerous and extremely likely to kill us all". I skimmed it (there's a lot of writing assuming grave danger) but found only the following example of AI lethality, and it appears fantastical:

> ...nanomachinery builds diamondoid bacteria, that replicate with solar power and atmospheric CHON, maybe aggregate into some miniature rockets or jets so they can ride the jetstream to spread across the Earth's atmosphere, get into human bloodstreams and hide, strike on a timer.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#46

Post by graeme »

crisdux wrote: post #32

Everything is turned into an existential risk these days. People are prone to believe in apocalyptic predictions because they fear the unknown and rely on flawed mental heuristics. Apocalyptic predictions automatically illicit skepticism from me.

Counterpoint: the more our capacity grows, the more we have the legitimate ability to bring absolute destruction

Plastic or some other chemical ending sperm counts —> we previously had no ability to affect the global ecosystem on this scale

Bioweapon virus or microorganism eviscerating civilization —> we had no prior ability to engineer such a plague

Nuclear holocaust —> we previously had no ability to destroy the world with weapons

Runaway climate change —> we previously had no ability to alter the climate

Ai sky net moment —> we previously had no ability to make such an intelligence

Solar flare cutting all electronics —> we previously had no dependence upon a single point of failure

I could go on. We’ve had plenty of civilizations collapse but nothing has been so linked before and we lacked such capacity before.

This is not to say any of the above will come to pass. But it is foolish to say “apocalypses were previously impossible hence such warnings can be dismissed out of hand”

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#47

Post by blowski »

crisdux wrote: post #32

Everything is turned into an existential risk these days. People are prone to believe in apocalyptic predictions because they fear the unknown and rely on flawed mental heuristics. Apocalyptic predictions automatically illicit skepticism from me.

Yes, this is how I feel about it. Between climate change, obesity, GM foods, mega-viruses, insects dying, extreme solar flares, and now AI, I’m surprised I’m still here anyway. Just perhaps AI will actually help solve some of those problems.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#48

Post by [deleted] »

skrebbel wrote: post #30
ctoth wrote: post #19

Earlier quoted context omitted.

Paul, can you point at anything at all other than your general vibes that would indicate that anyone from LessWrong intends to literally participate in mass suicide, as your link implies? Did I miss the big group buy of phenobarbital?

I’m not Paul, but this here has some serious Jonestown overtones: https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-annou... . It makes Yudkowski seem (to me) like the leader of a suicide cult more than anything else. The argument “yeah but what if he’s right?” is pretty much Pascal’s Wager by now, given how way out there the AI Doomers have gotten. I choose not to worry about the God from the Bible (ie I don’t…

[deleted]

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#49

Post by haswell »

luckylion wrote: post #33
moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

> We've had inquisitions historically, I would very much support a global AI inquisition. You know how inquisition usually goes. "You've been playing starcraft against 'the AI'. What do you have to say in your defense?" Maybe I'm just weird, but "a tech-based superintelligence emerges and decides to destroy humanity because it was in a bad mood" is something I'm totally fine with. Doom-Cults on the other hand I don't…

> Maybe I'm just weird, but "a tech-based superintelligence emerges and decides to destroy humanity because it was in a bad mood" is something I'm totally fine with.

It seems important to reflect on what you’re really saying here, and to ask how a) this is fine, and b) how “weird” sufficiently sums up the stance.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#50

Post by joshuahedlund »

lumenwrites wrote: post #20
PaulHoule wrote: post #6

There is "A.I. doomism" in general and there is also a specific cult-like group at the lesswrong website that really is an apocalyptic religion. It is astonishing how anything that deviates from its dogma in any way has to have "trigger warnings" and the senseless comments that users write on anything that suggests there could be intrinsic limits on intelligence. There ideology is such that a higher intelligence is g…

This is an inaccurate and ignorant comment that does not engage with the arguments people on LessWrong are making. If you have something smart to say about why these people are wrong in their beliefs, please do. Name calling a group of people a "cult" or "apocalyptic religion" is not an argument that contributes anything interesting to the conversation. If anyone is actually curious about what these people are saying…

> My lower-bound model of "how a sufficiently powerful intelligence would kill everyone, if it didn't want to not do that" is that it gets access to the Internet, emails some DNA sequences to any of the many many online firms that will take a DNA sequence in the email and ship you back proteins, and bribes/persuades some human who has no idea they're dealing with an AGI to mix proteins in a beaker, which then form a first-stage nanofactory which can build the actual nanomachinery. (Back when I was first deploying this visualization, the wise-sounding critics said "Ah, but how do you know even a superintelligence could solve the protein folding problem, if it didn't already have planet-sized supercomputers?" but one hears less of this after the advent of AlphaFold 2, for some odd reason.) The nanomachinery builds diamondoid bacteria, that replicate with solar power and atmospheric CHON, maybe aggregate into some miniature rockets or jets so they can ride the jetstream to spread across the Earth's atmosphere, get into human bloodstreams and hide, strike on a timer.

Does this kind of thing sound plausible to you? How much molecular biology have you studied? Even as someone who’s just been reading a genomics textbook for fun, the phrase “strike on a timer” alone is ludicrous. It sounds like someone who’s spent more time in science fiction. These are wildly speculative claims not at all grounded in believable science.

Post reply on HN