Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#491

Post by empath75 » Fri, Dec 23, 2016, 1:47 PM UTC

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

The real danger of ai is that they allow people to hide ethically dubious decisions that they've made behind algorithms. You plug some data into a system and a decision gets made and everyone just sort of shrugs their shoulders and doesn't question it.

Re: Superintelligence: The Idea That Eats Smart People

#492

Post by pc86 » Fri, Dec 23, 2016, 1:50 PM UTC

chrshawkes wrote: post #473

otakucode wrote: post #416

Earlier quoted context omitted.

Human beings have been extremely easy to kill for our entire existence. No system of laws can possibly keep you alive if your neighbors are willing to kill you, and nothing can make them actually unable to kill you. Your neighbor could walk over and put a blade in your jugular, you're dead. They could drive into you at 15MPH with their car, you're dead. They could set your house on fire while you're asleep, you're de…

That's not true, if I know my neighbors are coming I'm loading my Remington 870 and I'm waiting for that door to open. In America we're allowed to bear arms for protection.

You are missing the point. If everyone around you wants you dead and willing to do it, you're going to die. If the CIA knows of a terrorist camp and wants to kill the people there, they are going to die.

AI doesn't change this it just makes it easier.

Re: Superintelligence: The Idea That Eats Smart People

#493

Post by andrepd » Fri, Dec 23, 2016, 1:51 PM UTC

otakucode wrote: post #416

modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Human beings have been extremely easy to kill for our entire existence. No system of laws can possibly keep you alive if your neighbors are willing to kill you, and nothing can make them actually unable to kill you. Your neighbor could walk over and put a blade in your jugular, you're dead. They could drive into you at 15MPH with their car, you're dead. They could set your house on fire while you're asleep, you're de…

People don't kill each other, even if they want to, because they know if they do they will probably spend the rest of their lives in prison. What do you mean this doesn't dissuade people?

Re: Superintelligence: The Idea That Eats Smart People

#494

Post by andrepd » Fri, Dec 23, 2016, 1:54 PM UTC

pc86 wrote: post #492

chrshawkes wrote: post #473

Earlier quoted context omitted.

That's not true, if I know my neighbors are coming I'm loading my Remington 870 and I'm waiting for that door to open. In America we're allowed to bear arms for protection.

You are missing the point. If everyone around you wants you dead and willing to do it, you're going to die. If the CIA knows of a terrorist camp and wants to kill the people there, they are going to die. AI doesn't change this it just makes it easier.

Everyone wanted Bin Laden dead, but it still took a few years to manage that. So perhaps not as straightforward.

Re: Superintelligence: The Idea That Eats Smart People

#495

Post by andrepd » Fri, Dec 23, 2016, 1:55 PM UTC

falcolas wrote: post #485

nemof wrote: post #414

Earlier quoted context omitted.

truth. how we collectively became accepting of drones used by our governments to destroy targets half way round the world whilst the pilots sit in some skyscraper somewhere in our own countries is remarkable. honestly the disconnect is beyond deeply troubling.

There is no disconnect for those drone pilots, who become as troubled as anyone who killed someone with a lesser distance between.

Evidence for that?

Re: Superintelligence: The Idea That Eats Smart People

#496

Post by [deleted] » Fri, Dec 23, 2016, 1:58 PM UTC

rntz wrote: post #241

maxerickson wrote: post #213

Earlier quoted context omitted.

Einstein didn't look like a crank though. His papers are relatively short and are coherent, he either already had a PHD in physics or was associated with an advisor (I didn't find a good timeline; he was awarded the PHD in the same year he published his 4 big papers). Cranks lack formal education and spew forth the gobbledygook in reams.

By this measure, I would say Bostrom is not a crank. Yudkowsky is less clear. I'd say no, but I'd understand if Yudkowsky trips some folks' crank detectors.

[deleted]

Re: Superintelligence: The Idea That Eats Smart People

#497

Post by Sean1708 » Fri, Dec 23, 2016, 2:04 PM UTC

freshhawk wrote: post #386

apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

> but if the rule you use is "if they look like a crank, ignore everything they say", then you miss special relativity (and later general relativity) No you don't, you just don't catch it right away, relativity actually holds up under scrutiny. Besides, I reject the premise anyway. Einstein did serious work on the photoelectric effect first and then gradually worked towards relativity. Outside of the pop history he h…

Yep, Einstein was an expert in his field who wrote a couple of ground-breaking papers in his field. As far as I can tell no-one who is an expert in AI (or even similar fields) is worried at all about super-intelligence.

Re: Superintelligence: The Idea That Eats Smart People

#498

Post by lmm » Fri, Dec 23, 2016, 2:04 PM UTC

monk_e_boy wrote: post #176

Afforess wrote: post #159

> Observe that in these scenarios the AIs are evil by default, just like a plant on an alien planet would probably be poisonous by default. I believe this is a core misunderstanding. Bostrom never says that a superintelligent AI is evil by default. Bostrom argues the AI will be orthogonal, it's goals will be underspecified in such a way that leads it to destroy humanity. The paperclip optimizer AI doesn't want to kil…

An uncaring blind AI wouldn't be very interesting, I would assume whoever had turned it on and was observing it would just turn it off and tweek the algorithm. But I don't think this is even possible as when the AI was in child stage and was learning, it would learn from people, so it would become like people. Or at least understand them. At some point it has to know less than a human and will learn at a rate that hu…

> I don't agree with the assumption that a factory making paperclips will make the transition to a super intelligent AI in a short time frame.

Why? Because factories run by humans take a long time to transition? It might take weeks or months to make a process tweak in a human-run factory, but deploying new code happens in seconds or faster.

Re: Superintelligence: The Idea That Eats Smart People

#499

Post by ikeboy » Fri, Dec 23, 2016, 2:07 PM UTC

bambax wrote: post #389

daveguy wrote: post #249

Earlier quoted context omitted.

> If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe? That Elon Musk, Bill Gates and Stephen Hawking etc are all a little nutters when it comes to UFOs and aliens? I don't know a lot about UFOs and affiliated cults, but I'm going to guess that those mentioned are not UFO e…

> That Elon Musk, Bill Gates and Stephen Hawking etc are all a little nutters when it comes to UFOs and aliens? Yes! Elon Musk believes firmly that we're living in a simulation, and that doesn't make me believe in that theory more, it simply makes me admire Musk less. Just because someone is or has been extremely successful doesn't mean they're right about everything. Many successful and intelligent people have been…

That's one. If as many people who have spoken out about AI risk would also "firmly believe we're in a simulation", it would shift my views (I would expect there to be some evidence or very strong arguments for it).

Re: Superintelligence: The Idea That Eats Smart People

#500

Post by lmm » Fri, Dec 23, 2016, 2:08 PM UTC

otakucode wrote: post #405

He is completely skipping the most important part. The superintelligence has to have some reason to be in conflict with us. Human beings don't go out of their way to hunt down and eliminate ants. They don't find out what ants eat and sieze control of it to manipulate them. There is no reason to think that a superintelligent machine would be likely to present terrible interference to us proposed. So it's super smart a…

> We can reliably presume that it will need energy to achieve those goals. Will it need to achieve them quickly? Why?

Because a designer has told it to do this thing "as fast as possible", probably.

> Also, if we're going to build a brain and assume no quantum weirdness - why assume the total absence of a subjective morality? Why assume completely immunity to social influence, which the brains we observe most all encounter?

There is a lot more detail on this point in Bostrom's writings. Social influence is not the natural state of intelligence, it's something that has been built in by evolution. It won't be there in an AI unless we put it in there deliberately.

> having determined that the probability that some action it would take would result in its own destruction with a non-zero probability and that it is instead taking the safe route and not acting at all.

AI is all about making the best possible inferences from limited information - that's literally the whole point of the field. If there is no absolutely safe path it will go for the path it thinks is safest, and pay attention to what's happening. After all, turning itself off is very unlikely to result in its long-term survival.