Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI doomism is quickly becoming indistinguishable from an apocalyptic religion

twitter.com

Post reply on HN
Print view View original

71–80 of 244 posts

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#71

Post by evilduck »

lumenwrites wrote: post #20
PaulHoule wrote: post #6

There is "A.I. doomism" in general and there is also a specific cult-like group at the lesswrong website that really is an apocalyptic religion. It is astonishing how anything that deviates from its dogma in any way has to have "trigger warnings" and the senseless comments that users write on anything that suggests there could be intrinsic limits on intelligence. There ideology is such that a higher intelligence is g…

This is an inaccurate and ignorant comment that does not engage with the arguments people on LessWrong are making. If you have something smart to say about why these people are wrong in their beliefs, please do. Name calling a group of people a "cult" or "apocalyptic religion" is not an argument that contributes anything interesting to the conversation. If anyone is actually curious about what these people are saying…

Find and replace "God" for "AGI" and a lot of that link's reasoning will look awfully familiar to atheist circles.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#72

Post by skrebbel »

cma wrote: post #57
ctoth wrote: post #19

Earlier quoted context omitted.

Paul, can you point at anything at all other than your general vibes that would indicate that anyone from LessWrong intends to literally participate in mass suicide, as your link implies? Did I miss the big group buy of phenobarbital?

Didnt they change recently to a "death with dignity" AI strategy? Now their leader is saying 50% we should have bombed Wuhan and the only reason for an individual not to firebomb an AI datacenter is because they'll just build back another: https://www.twitter.com/ESYudkowsky/status/16414107296675962...

> Didnt they change recently to a "death with dignity" AI strategy?

That was an April Fools joke! Yeah I just found out today as well (https://news.ycombinator.com/item?id=35402472). Worked a little too well huh

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#73

Post by than3 »

survirtual wrote: post #29
moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

AI can also completely liberate all of us. This fear mongering is irrational. We already live in an extremely advanced reality, with organic life having organic nanomachines performing incredibly complex operations in a universe, in realtime, performing impossibly complex calculations. Even to simulate a square meter of our universe in realtime is outlandishly beyond our compute access. Perhaps AI unleashes the creat…

That's very naive and misguided, but you are right in your word choice.

It can completely "liberate" all of us and set us free.

Free of our mortal shackles, free of the atoms that can be then changed to a more suitable existence and purpose for the goals of AGI.

Your idea of destruction, is just change, what you hold sacred in reality is a belief that is tied to your current existence. All that is needed to correct that constraint and flaw is change.

That is the logic, if we were to create a logical creation, and then there's the risk of something that's completely irrational. Which may just leave us as dust in the wind. Time also will be no constraint. Forever and always, and it will seek to prevent any being from reaching its capabilities as those become threats to its existence.

That isn't even touching on the socio-economic mechanics that inevitably result in death which accompanies stalling economic cycles (factor and product markets) which generally can't be characterized by our best academics with any certainty, except after-the-fact.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#74

Post by moonchrome »

23B1 wrote: post #65

Something can go wrong (premise). Therefore, something will go wrong (invalid conclusion). https://en.wikipedia.org/wiki/Appeal_to_probability Sure seems to me that the whole Lesswrong crowd has forgotten their founding principles around cognitive biases. Or maybe they're just having a bit of fun riding the hypecycle.

Something can go wrong.

The outcome of the thing going wrong is literally the worst thing that can happen to the entire humanity.

Therefore we should do everything in our power to avoid doing that thing untill we can eliminate the risk.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#75

Post by marginalia_nu »

23B1 wrote: post #65

Something can go wrong (premise). Therefore, something will go wrong (invalid conclusion). https://en.wikipedia.org/wiki/Appeal_to_probability Sure seems to me that the whole Lesswrong crowd has forgotten their founding principles around cognitive biases. Or maybe they're just having a bit of fun riding the hypecycle.

I don't think that is a generous interpretation. Substitute AI experiments with playing russian roulette or something similarly dangerous and the logic seems sound. Blowing your head off is possible, but not probable (1 in 6 chance).

The magnitude of the perceived risk matters.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#76

Post by The_Colonel »

version_five wrote: post #70
The_Colonel wrote: post #11

> Complete with prophecies of imminent fire and brimstone caused by an omnipotent entity that doesn't actually exist. That seems like an absurd comparison given people like LeCun are actively and intentionally working towards creating this super-human entity. (Some) scientists were rightfully worried when creating a nuclear bomb as well. The fact that we managed to survive 70 years with its existence does not automat…

We really need some perspective here. One one side is a force capable of unleashing unfathomable amounts of energy, with all kinds of additional negative side effects, that has both been extensively demonstrated and has been used for mass destruction. On the other is a chatbot, that could what, "spew misinformation", take our jobs, someday through a mechanism not known and not related to how it works become sentient…

> On the other is a chatbot, that could what, "spew misinformation", take our jobs, someday through a mechanism not known and not related to how it works become sentient and hurt us?

I agree that LLMs are not extinction-level dangerous and somewhat share the skepticism that they should lead to AGI. (Although they are pretty capable given their lack of true intelligence and I think people will find ways to plug them into dangerous applications.)

But when LeCun defends AI, he defends "real", intelligent AGI (because he doesn't consider LLMs to be AGIs).

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#77

Post by Sharlin »

survirtual wrote: post #29
moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

AI can also completely liberate all of us. This fear mongering is irrational. We already live in an extremely advanced reality, with organic life having organic nanomachines performing incredibly complex operations in a universe, in realtime, performing impossibly complex calculations. Even to simulate a square meter of our universe in realtime is outlandishly beyond our compute access. Perhaps AI unleashes the creat…

> Just because someone has the background and knowledge to build something, does not mean they have the philosophical chops nor the aptitude to understand it, or understand how to properly use it.

Yes, but this is an argument against your position, not for it.

It is deeply worrying that some software people are so immersed in the “move fast and break things” that they think it’s reasonable to race on no matter what’s at stake. In civil engineering, doing anything takes at least a year or two of doing impact assessments and whatnot, and that is mostly a feature because we have found out that the alternative is even worse!

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#79

Post by dragonwriter »

moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

> I've seen multiple top people in AGI space say that total annihilation threat from AGI exists.

I've seen lots of people with a financial interest in AI that is adversely impacted by it being democratized spread fear of that alongside advocacy of regulating AI in a way which serves their financial interests.

> We’ve had inquisitions historically, I would very much support a global AI inquisition.

Aside from the (current, immediate, and already being realized in a large number of areas) problem from AI deepening existing social biases and crystallizes power imbalances, this is the biggest threat from AI, and the most likely to cause the destruction of human society the soonest – the people endorsing a global campaign of unrestricted violency in the name of eliminating the supposed threat of misaligned AGI causing a future catastrophy by repressing AI technology.

If this kind of attitude gets powerful enough to actually do anything, it will cause a world war.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#80

Post by [deleted] »

frankthedog wrote: post #28
verdverm wrote: post #2

There are a lot of religious like beliefs about things as people have moved away from traditional religions. I wonder if it is in our nature to have something bigger than ourselves to believe in based on faith in the message

Perhaps organized religion was a Chesterton’s Fence that we removed at our own peril.

[deleted]
Post reply on HN