Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#511

Post by generj » Fri, Dec 23, 2016, 2:52 PM UTC

andrepd wrote: post #494

pc86 wrote: post #492

Earlier quoted context omitted.

You are missing the point. If everyone around you wants you dead and willing to do it, you're going to die. If the CIA knows of a terrorist camp and wants to kill the people there, they are going to die. AI doesn't change this it just makes it easier.

Everyone wanted Bin Laden dead, but it still took a few years to manage that. So perhaps not as straightforward.

Not everyone wanted Bin Laden dead. Critically, the people he was hiding with and his organization as a whole very much wanted him alive. The people who actually surrounded him on a daily basis did not want to kill him.

Re: Superintelligence: The Idea That Eats Smart People

#512

Post by pc86 » Fri, Dec 23, 2016, 3:09 PM UTC

andrepd wrote: post #494

pc86 wrote: post #492

Earlier quoted context omitted.

You are missing the point. If everyone around you wants you dead and willing to do it, you're going to die. If the CIA knows of a terrorist camp and wants to kill the people there, they are going to die. AI doesn't change this it just makes it easier.

Everyone wanted Bin Laden dead, but it still took a few years to manage that. So perhaps not as straightforward.

How long between the time the CIA knew where he was (definitively) and the time he was killed?

Re: Superintelligence: The Idea That Eats Smart People

#513

Post by ahh » Fri, Dec 23, 2016, 3:14 PM UTC

fapjacks wrote: post #376

ahh wrote: post #315

Earlier quoted context omitted.

> Or give public schools $3K per student, > and you'll see huge changes. It's a lot of money. Yes, you'll see them spend $3K more per student. You won't actually see any improvements in the students, though. http://washington.cbslocal.com/2014/04/07/study-no-link-betw... The public schools have many problems; lack of money is not one of them. Having any idea what to do with it is.

Well, this is kind of a red herring, because it's widely known that schools of all types have been drastically increasing their administrative bodies, ballooning costs without actually doing anything for the students with that extra administration. Plus also it's the CATO Institute.

Think about what you're saying: "It's a red herring! We know that schools spend money they get on dumb things!" Yes, that's my point. :) If we could magically fix that, it might--in principle--become a good idea to give them more money. Giving them money does not actually magically fix that.

It's worth noting that Maciej forgot that the tech barons he hates actually tried this: https://www.washingtonpost.com/opinions/how-newark-schools-p...

Re: Superintelligence: The Idea That Eats Smart People

#514

Post by Tenhundfeld » Fri, Dec 23, 2016, 3:43 PM UTC

nemof wrote: post #414

coldtea wrote: post #395

Earlier quoted context omitted.

> The problem arises when non-state actors wreak havoc with drones and AI with inpunity. Well, for the rest of the world, state actors wreaking havoc with drones and AI with inpunity is already a problem.

truth. how we collectively became accepting of drones used by our governments to destroy targets half way round the world whilst the pilots sit in some skyscraper somewhere in our own countries is remarkable. honestly the disconnect is beyond deeply troubling.

I don't really grok this thinking. Why is destroying a target by drone different?

It doesn't seem substantively different from destroying a target via long-range missile or via a laser-guided bomb dropped from a human pilot flying thousands of feet over head. All three are impersonal ways of killing other human beings from a mostly-safe distance – especially in our modern asymmetrical engagements.

I agree that it feels gross to imagine a soldier sitting in a skyscraper pulling a trigger to kill people half way round the world, and it feels odd for someone to kill people in the morning and go home to sleep their bed that night, again and again, day after day. But military commanders have effectively been doing that since we've had faster-than-horse battlefield communication. So, I'm not convinced this is some brave new world of impersonal killing.

HOWEVER, I get the problem with drone warfare. Drones provide commanders with several benefits (no human casualties on "our" side, generally great accuracy, relatively low cost, etc.) that let them scale up the killing with minimal public outcry. This seems a real problem.

I guess my point is that the problem is not drones. The problem is killing so many people with so little oversight and so little apparent concern – whether by Cruise missile, drone, nanoswarm, T2, whatever.

I'd reword your sentence to be: "how we collectively became accepting of our governments casually killing our fellow humans half way round the world is remarkable"

Re: Superintelligence: The Idea That Eats Smart People

#515

Post by otakucode » Fri, Dec 23, 2016, 3:49 PM UTC

chrshawkes wrote: post #473

otakucode wrote: post #416

Earlier quoted context omitted.

Human beings have been extremely easy to kill for our entire existence. No system of laws can possibly keep you alive if your neighbors are willing to kill you, and nothing can make them actually unable to kill you. Your neighbor could walk over and put a blade in your jugular, you're dead. They could drive into you at 15MPH with their car, you're dead. They could set your house on fire while you're asleep, you're de…

That's not true, if I know my neighbors are coming I'm loading my Remington 870 and I'm waiting for that door to open. In America we're allowed to bear arms for protection.

Because you do not live in an action movie, you will not be able to stop them. You will not know they are coming, or when, or know how they intend to do it, etc. If you are well-defended with arms, they will simply use some other method. Having a shotgun does not make you significantly harder to kill in modern society. It would make you harder to kill if we were limited to cinematic tropes like declaring when we intend to do it, how we intend to do it, etc, sure, but we don't live in that world. We live in a world where the only thing that prevents our death, every day of our lives, is that no one nearby is willing to kill us.

Some people find that scary, and it may seem like a shaky thing to stake your life on. But, firstly, you have no choice. Secondly, it works for billions of people and has kept us safe for tens of thousands of years. Even after we developed the knowledge, tools, and ability to kill millions at the push of a button.

Re: Superintelligence: The Idea That Eats Smart People

#516

Post by novaleaf » Fri, Dec 23, 2016, 3:50 PM UTC

tboyd47 wrote: post #475

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

No doubt that his "inside arguments" have been rebutted extensively by the strong AI optimists and their singularity priests. After all, dreaming up scenarios in which robotic superintelligence dominates humanity is their version of saving the world. That's why I found the "outside arguments" here equally important and compelling. > The outside view doesn't care about content, it sees the form and the context, and it…

The posters rebuttals against the threat are totally under-thought cop-outs. For example, his first argument about "how would Hawking get his cat in a cage?" just put food in it. It's not hard to imagine an AI could come up with a similar motivation to get humans to do what it wants.

That's not to say that his general premise is wrong, but it's hard for me to take it seriously when his rebuttals are this weak.

Re: Superintelligence: The Idea That Eats Smart People

#517

Post by otakucode » Fri, Dec 23, 2016, 3:58 PM UTC

xendo wrote: post #418

otakucode wrote: post #405

He is completely skipping the most important part. The superintelligence has to have some reason to be in conflict with us. Human beings don't go out of their way to hunt down and eliminate ants. They don't find out what ants eat and sieze control of it to manipulate them. There is no reason to think that a superintelligent machine would be likely to present terrible interference to us proposed. So it's super smart a…

>So it's super smart and has its own goals. We can reliably presume that it will need energy to achieve those goals. Will it need to achieve them quickly? Why? Would the superintelligence be shortsighted enough to provoke humans into active combat against it? It happened so many times throughout human history, that it's really foolish to reject. People with their own goals burned, raped, murdered. Maybe superintellig…

>People with their own goals burned, raped, murdered.

Yes, they did those things to people. Because _they_ were people. Because people share the same needs for the same resources, and operate the same. That's all out the window for a machine-based superintelligence. Such an entity would not need our land, our food, our homes, our orifices, etc. I have no doubt it would have the ability to wipe us out. But I have never heard a cogent reason behind why it would need to. Especially considering that we would at least try to put up a fight and, if it were truly hopeless, likely do whatever would be the equivalent of 'salting the earth' before we were eliminated.

Re: Superintelligence: The Idea That Eats Smart People

#518

Post by idlewords » Fri, Dec 23, 2016, 4:06 PM UTC

thenomad wrote: post #461

apsec112 wrote: post #29

"Not many people know that Einstein was a burly, muscular fellow. But if Einstein tried to get a cat in a carrier, and the cat didn't want to go, you know what would happen to Einstein. He would have to resort to a brute-force solution that has nothing to do with intelligence, and in that matchup the cat could do pretty well for itself." This seems, actually, like a perfect argument going in the other direction. Ever…

There's no mention of iteration here, which is really what powers intelligence-based advantages. The first time Random Human A attempts to get Random Cat B into a box, they're going to have a hard time. They'll get there eventually, but they'll be coughing from the dust under the bed, swearing from having to lift the bloody sofa up, and probably have some scratches from after they managed to scare the cat enough for…

There's no need to reinvent solutions. The problem has already been given a proper mathematical treatment:

http://www-history.mcs.st-and.ac.uk/Extras/Spitzer_lion.html

Re: Superintelligence: The Idea That Eats Smart People

#519

Post by CydeWeys » Fri, Dec 23, 2016, 4:06 PM UTC

novaleaf wrote: post #516

tboyd47 wrote: post #475

Earlier quoted context omitted.

No doubt that his "inside arguments" have been rebutted extensively by the strong AI optimists and their singularity priests. After all, dreaming up scenarios in which robotic superintelligence dominates humanity is their version of saving the world. That's why I found the "outside arguments" here equally important and compelling. > The outside view doesn't care about content, it sees the form and the context, and it…

The posters rebuttals against the threat are totally under-thought cop-outs. For example, his first argument about "how would Hawking get his cat in a cage?" just put food in it. It's not hard to imagine an AI could come up with a similar motivation to get humans to do what it wants. That's not to say that his general premise is wrong, but it's hard for me to take it seriously when his rebuttals are this weak.

The emu "war" example is also similarly dumb. We know that humanity is very successful at wiping out animals even when we're not actively trying, just as a consequence of habitat encroachment or over-hunting. If you want to kill a bunch of emus, that can easily be done using appropriate methods. Having a cohesive military formation go after them in the huge Australian outback and giving up after a week is not the way to do so.

Re: Superintelligence: The Idea That Eats Smart People

#520

Post by p1esk » Fri, Dec 23, 2016, 4:07 PM UTC

Nyubis wrote: post #437

erikbye wrote: post #427

Earlier quoted context omitted.

I'm saying that's the pipe dream. Software won't change on its own.

We already have software that can change on its own. Neural nets are software too, and with things like backpropagation they can update their weights, essentially changing themselves. I'm not saying that this level of change is enough to get the disaster scenarios that Bostrom talks about, but it's a folly to say that self-changing software can't possibly exist.

Neural net changing its weights is no different from any other software updating the values of its variables.