Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

141–150 of 374 posts

Re: Among the A.I. doomsayers

#141

Post by CarRamrod »

nsainsbury wrote: post #98

We're not going to make it to anywhere close to AGI before we see widespread and systematic societal and environmental collapse on almost all fronts due to climate change. We're in the middle of the sixth mass extinction right now ( https://en.wikipedia.org/wiki/Holocene_extinction ), we're in unparalleled territory with ocean warming: ( https://climatereanalyzer.org/clim/sst_daily/ ) and as a society we are utterly…

You'll be glad to hear that the recent sudden jump in sea temperatures isn't caused by carbon dioxide. Turns out it just hasn't been windy enough in the Sahara lately (and environmentally friendly boat fuel may have contributed as well). https://heatmap.news/climate/why-is-the-atlantic-ocean-so-ho...

Re: Among the A.I. doomsayers

#142

Post by clooper »

HankB99 wrote: post #126
clooper wrote: post #74

Earlier quoted context omitted.

Lem wrote about this in Summa Technologiae. He called the field Intellectronics and explored what would happen once machine/electronic intelligence surpassed human intelligence: https://en.m.wikipedia.org/wiki/Summa_Technologiae

Not surprising that I'm not the first to think of this.

It's a very old trope in science fiction and most of the doom scenarios spelled out by the folks afraid of AI takeover is basically a waste of time. I would take the doomers more seriously if they actually brought some receipts from folks like Lem who were much better thinkers and much more adept at exploring the human condition and our fears of advanced machine intelligence.

Re: Among the A.I. doomsayers

#143

Post by preommr »

sberens wrote: post #131

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk

I am 90% sure that Altman's views on such issues are a machiavelian attempt at regulatory capture.

Re: Among the A.I. doomsayers

#145

Post by mitthrowaway2 »

clooper wrote: post #139
mitthrowaway2 wrote: post #129

Earlier quoted context omitted.

Keep following that line of thought, don't stop there! Why did those violent behaviors succeed in being selected for? Why didn't pacifism get selected for instead? Why does the government maintain a monopoly on violence, and why is it that the nth-degree consequence for violating societal laws is violence, instead of a monetary fine or a sternly-worded letter? Is there perhaps some kind of game-theoretic equilibrium…

AIs are immortal, they have no need for strategies developed by mortals like us. I spelled out a much better strategy that a sufficiently advanced AI could use to get rid of humans. It would simply encourage the most violent and self-destructive tendencies among humans and let us design our own demise simply because it would be the most sensible strategy for an intelligence that was not competing with the same resour…

A scenario in which an AI subtly and slowly guides us to our own doom by encouraging our inner demons still counts (in my mind at least) as an AGI doom scenario, just a slow-motion one. But it strikes me as very unlikely, because in allowing humans that much latitude we still pose a credible threat: we might instantiate a second AI that would compete with the first. There would have to be a large enough upside for it to be worth the risk. I'm not saying it certainly wouldn't happen the way you say, but it's by no means guaranteed.

Re: Among the A.I. doomsayers

#146

Post by i5heu »

sberens wrote: post #131

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk

I’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say about this topic is as relevant as something every other random person can say.

Re: Among the A.I. doomsayers

#147

Post by senordevnyc »

fwlr wrote: post #132
matteoraso wrote: post #31

I've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).

It’s not so much that there’s a little morality tag that randomly gets assigned the value of “nice” or “evil”, it’s more like there’s 1000 possible programs we would consider “super intelligent” and maybe it’s the case that 950 of them would reshape the world in a way we wouldn’t like - and when a powerful entity reshapes the world in a way we don’t like, we call that entity “evil”. ^ This is the basic reasoning behi…

[dead]

Re: Among the A.I. doomsayers

#148

Post by tivert »

sberens wrote: post #131

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk

So, is this extinction or not?

> We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like. - Sam Altman (https://blog.samaltman.com/the-merge)

I would say it is.

Re: Among the A.I. doomsayers

#149

Post by HKH2 »

ewhanley wrote: post #63
drooby wrote: post #62

AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

Mrs. Davis[0] offers a pretty compelling vision of this - absurd but not wholly unbelievable. The machine doesn’t necessarily have to supply the physical threat if it has humans who happily do its bidding. [0] https://en.m.wikipedia.org/wiki/Mrs._Davis

The hardest part is the AGI. People could easily sell their souls to the AGI devil to get knowledge and power, since if it truly is AGI, it should be able to make ideas compelling.

Re: Among the A.I. doomsayers

#150

Post by reducesuffering »

i5heu wrote: post #146
sberens wrote: post #131

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk

I’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say about this topic is as relevant as something every other random person can say.

Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure.

Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"

Post reply on HN