Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#361

Post by hackuser » Fri, Dec 23, 2016, 6:40 AM UTC

zevyoura wrote: post #123

modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

> Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

Currently there's a high cost to it. War is very expensive; even the richest country in the history of the world, the US, doesn't want to bear the costs. And it requires persuading masses of soldiers to go along with it.

Roboticized warfare might eliminate those constraints.

Re: Superintelligence: The Idea That Eats Smart People

#362

Post by ebspelman » Fri, Dec 23, 2016, 6:44 AM UTC

kobayashi wrote: post #279

mundo wrote: post #151

Earlier quoted context omitted.

People will stop downvoting this if you edit it to add a specific example or three.

Too late to edit, so I'll post just a few examples here: >The only way out of this mess is to design a moral fixed point, so that even through thousands and thousands of cycles of self-improvement the AI's value system remains stable, and its values are things like 'help people', 'don't kill anybody', 'listen to what people want'. Bostrom absolutely did not say that the only way to inhibit a cataclysmic future for hu…

>> "A machine intelligence need not share any of that complexity"

I think this is an assumption that both Bostrom and yourself would acknowledge as such.

We simply don't know enough about the nuances of intelligence to make an assertion that there are inherently varying kinds of it.

Re: Superintelligence: The Idea That Eats Smart People

#363

Post by r00fus » Fri, Dec 23, 2016, 6:47 AM UTC

mahyarm wrote: post #257

pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Calfornia is mismanged to hell. The bay area has some of the worst roads in the nation with very mild weather and a wealthy tax base. It cost $8billion to make 1 or 2 miles of central subway and only €11 billion to make the worlds longest tunnel under the alps. I have the same income tax rate as I did in canada yet there isnt universal healthcare and far more economic inequality. It goes on and on. If you trippled th…

You lost me on your rant about taxation without universal healthcare when the vast apportionment of that tax (and mis-spending like on the Iraq Occupation) is Federal not CA.

The problem is one of regulatory capture - corporate vested interests control the governance process and that means peons are getting less and less each day for their tax revenues.

Re: Superintelligence: The Idea That Eats Smart People

#364

Post by snowwrestler » Fri, Dec 23, 2016, 6:54 AM UTC

drewcrawford wrote: post #337

idlewords wrote: post #56

Earlier quoted context omitted.

I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.

To become the Tiger King you must overcome the entire population of tigers. To become the President, you need only overcome a thousand Florida voters. To intern the Japanese, you need only overcome two members of SCOTUS (Korematsu v. US) It isn't necessary for Hawking to be able to trick the average cat into a box. It's sufficient to trick a handful of cats in total.

This line of reasoning only works in hindsight. Working in real time, you won't know in advance how many Florida voters you will need to win, or which ones.

It's like saying that the way to overcome the population of tigers is to focus on raising and befriending the largest and most cunning tigers, who will then protect you against the rest. Ok, good idea, but unfortunately there is no way to know in advance which little tiger cubs will grow up to be the largest and most cunning.

Re: Superintelligence: The Idea That Eats Smart People

#365

Post by mdpopescu » Fri, Dec 23, 2016, 7:05 AM UTC

ovibos wrote: post #163

openasocket wrote: post #148

Earlier quoted context omitted.

I'll push back against the idea of smart factories leading to "a vast chasm between a new, tiny Hyperclass and the destitute masses." I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. Income inequality obviously benefits the rich (in that they by definition have more money), but only up to a point. We won't devolve…

Money is just a stand-in for resources and labor, and if automation makes labor very very cheap, the rich will only need the natural resources the poor sit on, not anything from the poor themselves.

Which means these alleged poor will be in the same position everyone on the planet is right now, without having this magic automation.

Re: Superintelligence: The Idea That Eats Smart People

#366

Post by devilsavocado » Fri, Dec 23, 2016, 7:07 AM UTC

Eliezer wrote: post #288

idlewords wrote: post #62

Earlier quoted context omitted.

I think this basic concept of intractability, which programmers are very familiar with, hasn't penetrated far enough into AI world. Bostrom and Yudkowsky in particular seem happy to hand-wave past computational complexity.

I'm getting kind of sick of people who haven't read 2% of the stuff imagining what they think we've never talked about over the last 16 years. https://intelligence.org/files/IEM.pdf

I haven't had time to read the entire thing, so I've skimmed over lots, but there doesn't seem to be much mention of complexity theory in the way I believe idlewords is talking about.

I'm very curious: What happens if the algorithms for general artificial intelligence, and the ability for an AI to improve itself are all NP-Hard problems? Is that covered?

It might be similar to the "intelligence combustion" scenario outlined. But that appears to be a scenario where do not need to be fearing superintelligence.

Re: Superintelligence: The Idea That Eats Smart People

#367

Post by Twisell » Fri, Dec 23, 2016, 7:10 AM UTC

fjdlwlv wrote: post #277

paulbaumgart wrote: post #211

Earlier quoted context omitted.

It just might be smart enough to figure out a way around that.

Naturally , Asimov already wrote about this. _The Last Question_ and _The Last Answer_ Really,Asimov is more on point and more fun to read than all these Superintenlligence pundits

He actually even wrote about the current debate. In his multiverse robots were actually banned from earth by bigots at some point.

I miss him I only started to read his book after his death. RIP Good doctor

Re: Superintelligence: The Idea That Eats Smart People

#368

Post by davedx » Fri, Dec 23, 2016, 7:19 AM UTC

TeMPOraL wrote: post #251

nine_k wrote: post #246

Earlier quoted context omitted.

I'm pretty sure that drone-tracking equipment is mounted all over NYC, and more is being deployed as we talk. I can also imagine that a rouge drone arriving from afar and large enough to carry a bomb will be shot down by police if it can't be identified. If it does not carry a bomb, police will apologize.

I'm not buying that. In fact, the only reason such drone attacks don't happen, or why people hadn't been casually blowing each other up for the past five decades with explosives attached to RC cars / planes is that in general, people are nice to each other . There's plenty of tools out there for dedicated people to wreck havoc in populated areas. Such people simply are very, very rare.

Also as recent news from Berlin shows, you don't even need a drone, a regular old van will do

Re: Superintelligence: The Idea That Eats Smart People

#369

Post by Ygg2 » Fri, Dec 23, 2016, 7:30 AM UTC

modeless wrote: post #177

zardo wrote: post #156

Earlier quoted context omitted.

>There is a limit to how far a soldier will go, ethically. What horrible things to you have in mind that armies have not already done?

It's not that AIs will do worse things than humans have already done. It's that AIs will do those terrible things much more efficiently and with much lower risk to the people in charge.

On the plus side - no more rape.

Re: Superintelligence: The Idea That Eats Smart People

#370

Post by [deleted] » Fri, Dec 23, 2016, 7:30 AM UTC

modeless wrote: post #177

zardo wrote: post #156

Earlier quoted context omitted.

>There is a limit to how far a soldier will go, ethically. What horrible things to you have in mind that armies have not already done?

It's not that AIs will do worse things than humans have already done. It's that AIs will do those terrible things much more efficiently and with much lower risk to the people in charge.

[deleted]