Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

151–160 of 374 posts

Re: Among the A.I. doomsayers

#151

Post by i5heu »

reducesuffering wrote: post #150
i5heu wrote: post #146

Earlier quoted context omitted.

I’d like to remind people that these ppl have no more knowledge about AGI as anyone else on this planet since there is no knowledge yet and everything they say about this topic is as relevant as something every other random person can say.

Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"

They did not build a AGI yet so they have build as many AGIs as anyone else and therefore they are also laypersons regarding the effects of AGI on humanity.

Re: Among the A.I. doomsayers

#152

Post by blibble »

dkjaudyeqooe wrote: post #20
jodrellblank wrote: post #18

Earlier quoted context omitted.

Making the Earth uninhabitable to all humans is bad news for Putin, a human who lives on Earth. He derives his power from hundreds of millions of people in Russia working, and his lifestyle from hundreds of millions of people around the world buying Russian exports so an attack on everyone is also going to harm him. AI wouldn’t be human, it wouldn’t necessarily object to a world uninhabitable to humans, or be hurt by…

OK, so how does AI get its hands on nuclear weapons?

if trump can figure it out an AI probably could

Re: Among the A.I. doomsayers

#153

Post by bamboozled »

p1esk wrote: post #84
HankB99 wrote: post #49

If I let my imagination run wild, I can imagine that at some point AI becomes in some way sentient. By that I mean it gains reasoning, some sort of understanding, and motivation. I wonder what the possibility is that this AI will decide that a pitched battle is going to waste resources and risk humans pulling the plug. What if it understood that and instead operated so subtly that it was not obvious it was controllin…

If AI does ever become sentient, it will probably reason and decide about eliminating large swaths of human population in a similar fashion as we reason and decide about eliminating large swaths of ant population.

I’ve personally never ever considered killing large swathes of ants

Re: Among the A.I. doomsayers

#154

Post by clooper »

mitthrowaway2 wrote: post #145
clooper wrote: post #139

Earlier quoted context omitted.

AIs are immortal, they have no need for strategies developed by mortals like us. I spelled out a much better strategy that a sufficiently advanced AI could use to get rid of humans. It would simply encourage the most violent and self-destructive tendencies among humans and let us design our own demise simply because it would be the most sensible strategy for an intelligence that was not competing with the same resour…

A scenario in which an AI subtly and slowly guides us to our own doom by encouraging our inner demons still counts (in my mind at least) as an AGI doom scenario, just a slow-motion one. But it strikes me as very unlikely, because in allowing humans that much latitude we still pose a credible threat: we might instantiate a second AI that would compete with the first. There would have to be a large enough upside for it…

I just think it is a little odd that all scenarios involve conflict when a much simpler strategy for an immortal AI is to just sit back and let people do what they do to guarantee an outcome in the AIs favor. Too much intelligence is not the problem we are facing, the real problem is lack of it.

Re: Among the A.I. doomsayers

#155

Post by reducesuffering »

tivert wrote: post #148
sberens wrote: post #131

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk

So, is this extinction or not? > We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like. - Sam Altman ( https://blog.samaltman.com/the-merge ) I would say it is.

Even worse, some factions literally advocate for killing all humans in the pursuit of a synthetic intelligence, and YC's Garry Tan is advocating for these people!

Beff Jezos (e/acc founder): "e/acc has no particular allegiance to the biological substrate for intelligence and life, in contrast to transhumanism

    Parts of e/acc (e.g. Beff) consider ourselves post-humanists; in order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates"
https://beff.substack.com/p/notes-on-eacc-principles-and-ten...

Re: Among the A.I. doomsayers

#156

Post by tivert »

13years wrote: post #117
drooby wrote: post #62

AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

Potentially there are no good outcomes even if AGI remains under control. If we are actually able to create the type of AI entity they are trying to create, one that exponentially improves, the implications are beyond what most have thought about. Something I recently wrote in depth about here. How we completely misunderstand the future that we think will happen. "The implication is that everyone is enthusiastically…

> https://www.mindprison.cc/p/the-technological-acceleration-p...

That is a very good and insightful article. One of the biggest issues I have with AI enthusiasts is that they talk about creating these incredible capabilities ... then they imagine the resulting world with them will look like some hackneyed sci-fi work, like Star Wars, whose universe seems like it could be kinda cool to live in until you actually think about it and realize it's all fridge logic and nothing about it actually makes sense (e.g. WWII-style dogfights in the vacuum of space).

The only issue with the article is sometimes it's too optimistic, e.g.

> No one will care about you or what you create as there is nothing you can offer they can’t simply wish for themselves. Life is all about you and the machine, nothing else. We land in a world dominated by techno-dystopian narcissism.

> ...

> Whatever may be the current direction of society under the present technological capabilities, we can only assume AI will accelerate society towards the path we currently are traveling which is toward a techno dystopia of a populace mesmerized by shiny glimmering lights.

I kind of find it hard to believe there would be any stability for any people suck in that state, I think they'd instead be "cleaned up" and disposed of in relatively short order. Those people would be utterly useless to whatever still has power in society, so why service and protect them indefinitely?

Re: Among the A.I. doomsayers

#157

Post by reducesuffering »

i5heu wrote: post #151
reducesuffering wrote: post #150

Earlier quoted context omitted.

Yes, let's go with the random layperson knowledge of an HN commenter compared to the people smart enough to actually build all the AGI tech. 50/50 coin toss, I'm sure. Dario Amodei (Anthropic CEO, builder of Claude 3 Opus): "My chance that something goes, you know, really catastrophically wrong on the scale of human civilization, might be between 10 - 25%"

They did not build a AGI yet so they have build as many AGIs as anyone else and therefore they are also laypersons regarding the effects of AGI on humanity.

What do you call something humans built (artificial) that can solve all topics of AP tests (general) with most being 5/5, better than almost all humans (intelligence)?

Re: Among the A.I. doomsayers

#158

Post by bamboozled »

tivert wrote: post #148
sberens wrote: post #131

I'd like to remind people of this single-sentence statement: > Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war. [0] Signed by Demis Hassabis, Sam Altman, and Bill Gates among others [0] https://www.safe.ai/work/statement-on-ai-risk

So, is this extinction or not? > We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like. - Sam Altman ( https://blog.samaltman.com/the-merge ) I would say it is.

Sam Altman really seems unhinged, or he just says this stuff to be edgy...or both.

Re: Among the A.I. doomsayers

#159

Post by mitthrowaway2 »

clooper wrote: post #154
mitthrowaway2 wrote: post #145

Earlier quoted context omitted.

A scenario in which an AI subtly and slowly guides us to our own doom by encouraging our inner demons still counts (in my mind at least) as an AGI doom scenario, just a slow-motion one. But it strikes me as very unlikely, because in allowing humans that much latitude we still pose a credible threat: we might instantiate a second AI that would compete with the first. There would have to be a large enough upside for it…

I just think it is a little odd that all scenarios involve conflict when a much simpler strategy for an immortal AI is to just sit back and let people do what they do to guarantee an outcome in the AIs favor. Too much intelligence is not the problem we are facing, the real problem is lack of it.

That's a nice thought, but if you were the AI, and you'd just seen humans create a superintelligent AI (yourself) with some goal (eg. making paperclips), how long do you expect you can peacefully wait and watch before they make a second immortal AI that only cares about making pencil crayons, who you'll then have to share the Earth's finite resources with for the rest of time? Are there any actions available to you that might lessen the odds of that bad outcome and improve the odds of achieving your own goals? Are there strong reasons not to take those actions?

I agree that we're facing a lack of intelligence; I think if we had more (or perhaps, more wisdom), then we would know we shouldn't be building such things in the first place.

Re: Among the A.I. doomsayers

#160

Post by Animats »

drooby wrote: post #62

AGI escaping a sandbox is truly terrifying. There will be a subgroup of the population that will worship it and work for it. It's not so much AGI that scares me - it's the humans I'm scared of.

> AGI escaping a sandbox is truly terrifying.

That's already happened. The AGI's are called corporations. Most governments have failed to regulate corporations successfully, and have been unable to keep them from becoming almost powerful enough to challenge governments.

The accelerationists for corporations were a group of economists, led by Milton Friedman, and a group of business leaders, organized by the U.S. Chamber of Commerce. In the 1970s and 1980s, they pushed the ideas that corporations are responsible only to their stockholders, and that government should not interfere with the concentration of corporate power. Those were not mainstream ideas of the 1930s to 1960s. The corporate accelerationists succeeded. That's when corporations escaped the sandbox.

Now that's an "alignment problem".

Post reply on HN