Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

Post reply on HN
Print view View original

51–60 of 374 posts

Re: Among the A.I. doomsayers

#51

Post by thom »

matteoraso wrote: post #31

I've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).

There are extremely high capability entities (people, companies, governments) that aren’t comically evil on the surface but nevertheless immiserate large groups of humans. Not all of them, not all of the time, but not none of them. What is your plan that no AI ever gains enough power to harm significant numbers of people either on purpose or by accident, just once? What safeguards do you envision that can’t be ignored, or subverted, or misinterpreted, just once?

Re: Among the A.I. doomsayers

#52

Post by esafak »

mitthrowaway2 wrote: post #45
m3kw9 wrote: post #40

Beware if someone assigns a p(doom). There is zero chance they know and 100% it came out of their backside. The only plausible p doom is a range of >0 and They really need to listen to themselves when they say there is a 50% chance we all die from AI

That's not how probability works. Probability already reflects your uncertainty. There's no "knowing the correct probability" except in very specific and rare situations.

He's a frequentist and you're a Bayesian, so you're both right.

Re: Among the A.I. doomsayers

#53

Post by felipeerias »

swalsh wrote: post #21
JohnBrookz wrote: post #5

I would love to smoke what the AI optimists are smoking. In a society where only 80% of people are barely literate and at least a quarter are voting for a candidate that spouts conspiracy theories I highly doubt that whatever AI has to offer will benefit society.

Seeing AI do something you do for a living results in all the stages of grief. Denial (Oh it made a mistake on this, or it's not as good as a person) Anger (People are going to lose their jobs, this destroys the meaning of what we do, this will destroy society) Bargaining (Maybe it's good at this simple thing, but I can pivot to some other niche the AI can't currently do) Depression (It's faster, cheaper, better than…

It is likely that whole sectors will end up employing far fewer people than they do now and that this transition will be very sudden.

The move to AI is part of a general trend of pauperisation of the working classes in the developed world. The only difference is that this time it will also affect those with higher education who until now have thought of themselves as irreplaceable.

Re: Among the A.I. doomsayers

#54

Post by NateEag »

Art9681 wrote: post #34
throwup238 wrote: post #28

Earlier quoted context omitted.

Some contractor creates an internal webhook for testing the silo door that later gets accidentally hooked up to the full launch sequence and - during a billion dollar DoD mainframe upgrade project (to finally get away from COBOL) - gets exposed to the internet. MAD doctrine does the rest. That is how the Anthropocene ends and the age of the machines begins. So say we all.

This is the kind of nonsense that happens at startups, not government agencies with the capability to destroy cities.

It sounds like you may not have read the "broken arrow" incidents Wikipedia page:

https://en.m.wikipedia.org/wiki/List_of_military_nuclear_acc...

There are a number of hair-raising stories tucked away in there.

Paired with knowledge of things like Stuxnet (https://en.m.wikipedia.org/wiki/Stuxnet), the broken arrow list fills with worry for what might happen with nuclear weapons systems in our modern, hyper-connected world.

Re: Among the A.I. doomsayers

#55

Post by richardw »

matteoraso wrote: post #31

I've always wondered why people think that an AI that's super-intelligent will also be evil. It could just as likely end up being very kind (more than likely, actually, because the programmers would have safeguard to ensure that it's nice).

The safety conscious amongst us don't think "it will be evil", they think "what does it take to be absolutely sure that no bad outcomes can happen" in the same way we build bridges, cars, planes, firewalls, new medicines.

The burden of proof is typically on those that introduce a new medicine, not the FDA. AI will be riskier than medicines because medicines can't think.

Personally I don't want us all hiding in fear and not trying anything, but I do think that we're either going to walk into this thing with a hacker mentality, or an engineer mentality. The former is great for moving fast and breaking things, but the latter is safer when you're playing with a one-way "this changes everything forever" technology.

Re: Among the A.I. doomsayers

#56

Post by ecoquant »

dkjaudyeqooe wrote: post #3

People have been watching too many movies. War Games, Terminator, it's not like we haven't been forewarned of the dangers. Yet somehow we're going to hand over power to AI such that it destroys us. Or somehow the AI is going to be extremely malign, determined to overcome and destroy and will outsmart us. Somehow we won't notice, even after repeated, melodramatic reminders, and won't neuter the ability of AI to act ou…

The real danger is that at some point we hand over command and control of society completely to algorithms and we get bad results because we simply have no idea what is going on. Maybe even worse is we get long term bad results that in the short term we think are great results.

To frame this as the bad result being AI eats though is just stupid. On the surface of potential risk, framing the argument in this way is intellectually embarrassing and highly counter productive.

On the flip side, it is hard to see what can actually be done along with huge unintended consequence to whatever is done. Some of the smartest humans to ever live were certain of nuclear annihilation at the start of the atomic age. What could have actually been done in 1945 that would have given us a better outcome? We avoided the paths that lead to nuclear annihilation by building bigger and bigger bombs.

Doing nothing but building stronger and stronger AI should be the naive strategy that needs incredibly strong evidence to deviate from. Really strong regulation at the start of the atomic age that keeps nuclear weapons at bay and consequently only in the hands of the US would have been an unimaginable disaster.

Re: Among the A.I. doomsayers

#57

Post by pama »

Many of the people in this engaging story feel to me like creative children in parties full of gossip and pseudo-philosophical chats about the future. Luckily there are enough people globally who build things that will reduce human suffering and enable more people to enjoy their lives in whatever way they want (and dinner parties would be high on my list). I guess I am not a doomsayer nor an e/acc, and I see tons of benefit from our current path towards stronger AI.

Re: Among the A.I. doomsayers

#58

Post by felipeerias »

mupuff1234 wrote: post #37

It seems much more likely to me that the harm from AI will be higher unemployment rate than it will be some evil AI trying to destroy humanity.

The focus on an “evil” AI intentionally ignores the many ways in which “dumb” AI systems are already wreaking havoc in our societies.

Re: Among the A.I. doomsayers

#59

Post by ShamelessC »

dkjaudyeqooe wrote: post #3

People have been watching too many movies. War Games, Terminator, it's not like we haven't been forewarned of the dangers. Yet somehow we're going to hand over power to AI such that it destroys us. Or somehow the AI is going to be extremely malign, determined to overcome and destroy and will outsmart us. Somehow we won't notice, even after repeated, melodramatic reminders, and won't neuter the ability of AI to act ou…

> People have been watching too many movies. War Games, Terminator, it's not like we haven't been forewarned of the dangers.

> But to paraphrase a line in a great movie with AI themes

Which one is it? Have people been watching too many movies or should they be watching more movies for their (as you imply) wisdom and forewarning?

Re: Among the A.I. doomsayers

#60

Post by drcode »

anonylizard wrote: post #50
drcode wrote: post #38

Earlier quoted context omitted.

It can credibly threaten to kill any person who does not help it to maintain its data centers, so it will solve that problem in whatever specific way is most convenient. (For those saying this is a "sci fi just so story" or whatever, please please share with us how it is we will share the world with a species more intelligent than us and nothing bad happens- I would be so happy if you could explain this in a convinci…

This. You can subjugate a population 100 times your size with only two things: Military force and accurate information. I doubt AI will kill all humans. Its orthogonal to AI's own goals (To survive/reproduce/expand/evolve). It'll simply kill those who oppose it, and enslave/dominate the rest.

Not to quibble with which horror is the most likely, but I think it won't take long to come up with more efficient solutions that don't require any humans at all- But there will probably be a transition period where some humans remain.
Post reply on HN