Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI doomism is quickly becoming indistinguishable from an apocalyptic religion

twitter.com

Post reply on HN
Print view View original

81–90 of 244 posts

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#81

Post by serverholic »

jszymborski wrote: post #51

As always, understanding of nuanced issues will pool at the extremes. It is both true that: - Adoption of DL allows corps to launder or inadvertently adopt unethical practices that exacerbate inequalities. - We have not created AGI, LLMs are not AGI, stable diffusion is not AGI and none of these fields of research are indication we are closer to AGI. This is like thinking Clever Hans is the portent of our future equi…

I’m curious why you are so confident in your assertion. It seems to me that an advanced statistical model of the world is an essential component of AGI. How do you know that we aren’t a few breakthroughs away from AGI?

Some recent papers have shown significant performance improvements when these models are allowed to respond to their own outputs.

How do you know that putting an LLM in a fancy loop with access to external memory and tools isn’t AGI?

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#82

Post by deschutes »

23B1 wrote: post #65

Something can go wrong (premise). Therefore, something will go wrong (invalid conclusion). https://en.wikipedia.org/wiki/Appeal_to_probability Sure seems to me that the whole Lesswrong crowd has forgotten their founding principles around cognitive biases. Or maybe they're just having a bit of fun riding the hypecycle.

Who knows. It's completely uncharted territory. It's not hard to imagine how malevolent AGI could end us.

The illusion of control over something that may well have incomprehensible intelligence is a joke. Tell me, what control does the farm animal have over its own fate?

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#83

Post by eternalban »

cwmoore wrote: post #45
lumenwrites wrote: post #20

Earlier quoted context omitted.

This is an inaccurate and ignorant comment that does not engage with the arguments people on LessWrong are making. If you have something smart to say about why these people are wrong in their beliefs, please do. Name calling a group of people a "cult" or "apocalyptic religion" is not an argument that contributes anything interesting to the conversation. If anyone is actually curious about what these people are saying…

That link is not at all a list of "why AI is extremely dangerous and extremely likely to kill us all". I skimmed it (there's a lot of writing assuming grave danger) but found only the following example of AI lethality, and it appears fantastical: > ...nanomachinery builds diamondoid bacteria, that replicate with solar power and atmospheric CHON, maybe aggregate into some miniature rockets or jets so they can ride the…

https://news.ycombinator.com/item?id=35402539

mind viruses.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#84

Post by moonchrome »

survirtual wrote: post #29
moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

AI can also completely liberate all of us. This fear mongering is irrational. We already live in an extremely advanced reality, with organic life having organic nanomachines performing incredibly complex operations in a universe, in realtime, performing impossibly complex calculations. Even to simulate a square meter of our universe in realtime is outlandishly beyond our compute access. Perhaps AI unleashes the creat…

> If it happens, there is no stopping it anyway.

Hopefully this isn't the case - AI development requires a lot of data/compute - societally we could block access to these things and prevent further development in that direction.

There are only a few places in existence capable of producing HW required or in possession of it. I'm fine with stopping compute progress for however long it takes to deal with safety issues. Society isn't perfect now but the level of compute we've had available for years is sufficient for generating a lot more value - we can afford to put AI research on ice.

I just don't get it - so much attention is given to things like climate change - this is a mild inconvenience compared to AGI threat.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#86

Post by userforcomment »

The_Colonel wrote: post #11

> Complete with prophecies of imminent fire and brimstone caused by an omnipotent entity that doesn't actually exist. That seems like an absurd comparison given people like LeCun are actively and intentionally working towards creating this super-human entity. (Some) scientists were rightfully worried when creating a nuclear bomb as well. The fact that we managed to survive 70 years with its existence does not automat…

I think nuke analogy is very tempting since US banned A100 export to China. One could assume that AI race of 21st century ~ nuke race in 20th. Note that Nvidia circumvented ban, so race is on..

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#87

Post by akvadrako »

dragonwriter wrote: post #79
moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

> I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I've seen lots of people with a financial interest in AI that is adversely impacted by it being democratized spread fear of that alongside advocacy of regulating AI in a way which serves their financial interests. > We’ve had inquisitions historically, I would very much support a global AI inquisition. Aside from the (cur…

Both can be true at the same time. I see it likely that:

1. Superintelligent AGI seems inevitable, even if it takes thousands of years. So fighting against it requires destroying all progress and freedom.

2. But if it comes we are no longer the dominant species; we'll have as much control as big animals do now. If that isn't essentially being destroyed, it's close enough.

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#88

Post by moonchrome »

dragonwriter wrote: post #79
moonchrome wrote: post #15

I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I think I've heard Karpathy say something along the lines of worst case scenario of AGI is worse than a nuclear armageddon on Lex Friedman podcast. Honestly the fear of AGI worst-case scenario is worse than any other fear I've experienced. I've accepted the fact that I can die a long time ago, I was afraid but mortality is…

> I've seen multiple top people in AGI space say that total annihilation threat from AGI exists. I've seen lots of people with a financial interest in AI that is adversely impacted by it being democratized spread fear of that alongside advocacy of regulating AI in a way which serves their financial interests. > We’ve had inquisitions historically, I would very much support a global AI inquisition. Aside from the (cur…

> I've seen lots of people with a financial interest in AI that is adversely impacted by it being democratized spread fear of that alongside advocacy of regulating AI in a way which serves their financial interests.

Given the rate of progress, shitshow management of basic alignment issues and the incentives that lead to those decisions - what's your argument that this isn't probable? You're betting that it just can't be developed ?

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#89

Post by luckylion »

haswell wrote: post #49
luckylion wrote: post #33

Earlier quoted context omitted.

> We've had inquisitions historically, I would very much support a global AI inquisition. You know how inquisition usually goes. "You've been playing starcraft against 'the AI'. What do you have to say in your defense?" Maybe I'm just weird, but "a tech-based superintelligence emerges and decides to destroy humanity because it was in a bad mood" is something I'm totally fine with. Doom-Cults on the other hand I don't…

> Maybe I'm just weird, but "a tech-based superintelligence emerges and decides to destroy humanity because it was in a bad mood" is something I'm totally fine with. It seems important to reflect on what you’re really saying here, and to ask how a) this is fine, and b) how “weird” sufficiently sums up the stance.

I'm fine with humans having outcompeted the neanderthals and whatever else there was. I'm just as fine with someone else doing it to us, at least if it's someone or something that is smarter than us and not some virus or bacteria that kills us all (which would be kind of lame). If an artificial superintelligence wants to take over the flame of progress, I'm fine with that.

But I'm also totally fine with considering that stance weird (or whatever word you'd prefer). I'm aware that others view these things very differently, just as others are much, much more worried about their individual demise than I am worried about mine (or theirs).

Re: AI doomism is quickly becoming indistinguishable from an apocalyptic religion

#90

Post by akvadrako »

mitthrowaway2 wrote: post #34
survirtual wrote: post #29

Earlier quoted context omitted.

AI can also completely liberate all of us. This fear mongering is irrational. We already live in an extremely advanced reality, with organic life having organic nanomachines performing incredibly complex operations in a universe, in realtime, performing impossibly complex calculations. Even to simulate a square meter of our universe in realtime is outlandishly beyond our compute access. Perhaps AI unleashes the creat…

"Things might be fine"? This doesn't seem like a sufficiently convincing take? It takes 20 years of due diligence and environmental impact studies just to build a single hydroelectric dam, if not 50. Why should our attitude toward AI be to shrug and hope for the best?

It's easy to stop a dam. It's in a physical location controlled by one country.
Post reply on HN