Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#201

Post by Koshkin » Fri, Dec 23, 2016, 1:27 AM UTC

Intelligence, having been born as a biological defense mechanism, is evil. It is a weapon. For an intelligent individual, being (or, rather, appearing) "good", is just another layer of defense - this time against other, likewise intelligent, individuals. "An armed society is a polite society."

Spying is a form of intelligent behavior. ("Intelligence.") So is stealing (and not getting caught, if possible). And, no doubt, hacking [1] is, too.

Wouldn't absolute intelligence mean absolute evil, then?

[1] http://www.reuters.com/article/us-cyber-ukraine-idUSKBN14B0C...

Re: Superintelligence: The Idea That Eats Smart People

#202

Post by cbr » Fri, Dec 23, 2016, 1:29 AM UTC

ajamesm wrote: post #129

apsec112 wrote: post #118

Earlier quoted context omitted.

"how have cell phones "dramatically improved" the lives of the poor?" There's tons of stuff on this, but, eg., here's a poster from USAID: https://s-media-cache-ak0.pinimg.com/originals/09/35/2d/0935...

Did you look at these metrics? They're weak as fuck. Most of them are "there's a platform or chart for this now" which is meaningless and probably intentionally decontextualized. To be clear, it's good that people have access to the internet and all the interconnectivity that it brings. I'm not slamming the basic premise. What I'm angry about is the opportunity cost. Telecoms & ISPs roll out bare minimums and say "he…

This isn't the US, but cell phones make M-Pesa possible, which is very positive: http://www.jefftk.com/suri2014.pdf http://www.jefftk.com/suri2016.pdf

Re: Superintelligence: The Idea That Eats Smart People

#203

Post by hawkice » Fri, Dec 23, 2016, 1:32 AM UTC

idlewords wrote: post #196

seiferteric wrote: post #147

Earlier quoted context omitted.

I was really surprised when I read the quote, they were worried about a runaway fusion reaction! I had heard this story before but always thought they were worried about "burning" nitrogen in the chemical sense, ie. creating nitrogen oxides. It seems doubly ridiculous to worry about fusion when we can't even get a fusion reactor to work.

It wasn't a ridiculous thing to worry about at all.

Yeah, I'm with the scientists on this. They had built the first bombs only a couple years after the first nuclear pile (a sustained nuclear chain reaction), and it was clear what they were trying to do was massively in excess of that. Taking a moment to make sure Earth didn't become a nuclear pile would be great.

n.b. That first nuclear pile was in December 1942, which is way later than even I thought it was. This stuff was happening fast.

Re: Superintelligence: The Idea That Eats Smart People

#204

Post by zdean » Fri, Dec 23, 2016, 1:34 AM UTC

zevyoura wrote: post #123

modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

I think they're largely limited by the greater public's threshold for acceptable casualties of their own soldiers during war. What the AI'ing of war does is to remove that natural limit and raise the amount of death and destruction a military can inflict without pushback from the general population.

Re: Superintelligence: The Idea That Eats Smart People

#205

Post by incompatible » Fri, Dec 23, 2016, 1:38 AM UTC

k8t wrote: post #72

Let's take a step back and look at the meta-level. So there's maybe a chance AI is extremely dangerous (as in wipes humanity out), and there's a chance that AI might not be dangerous. There are arguments between how likely each choice is. But the more important fact is that since there's so much at stake (all of humanity), we should likely be really really really sure about things. For example, let's say you live in…

In practice humans don't usually care about such arguments. They'll carry on as usual until something bad actually happens. I'm sure there are plenty of crippled people who regret not wearing a seatbelt, or people with skin cancer who regret not wearing a hat. Not to mention situations on a larger scale like building a lot of nuclear weapons or increasing the C0_2 level in the atmosphere.

Re: Superintelligence: The Idea That Eats Smart People

#206

Post by tim333 » Fri, Dec 23, 2016, 1:41 AM UTC

>Hopefully you'll leave this talk a little dumber than you started it, and be more immune to the seductions of AI that seem to bedevil smarter people.

I rather like the smarter people stuff - it's kind of exciting to figure how AI could solve human problems and stuff like the OpenAI initiative seem a sensible precaution against things going wrong. The arguments against seem a little reminiscent of arguments against global warming that say what do all these 'experts' know? I can look out the window and it's snowing as much as ever.

Re: Superintelligence: The Idea That Eats Smart People

#207

Post by seiferteric » Fri, Dec 23, 2016, 1:42 AM UTC

idlewords wrote: post #196

seiferteric wrote: post #147

Earlier quoted context omitted.

I was really surprised when I read the quote, they were worried about a runaway fusion reaction! I had heard this story before but always thought they were worried about "burning" nitrogen in the chemical sense, ie. creating nitrogen oxides. It seems doubly ridiculous to worry about fusion when we can't even get a fusion reactor to work.

It wasn't a ridiculous thing to worry about at all.

didn't really mean it was ridiculous for them to think about, just that it seems so now in retrospect.

Re: Superintelligence: The Idea That Eats Smart People

#208

Post by [deleted] » Fri, Dec 23, 2016, 1:43 AM UTC

kazagistar wrote: post #189

They always miss a critical and subtle assumption: that intelligence scales equal to or faster then the computational complexity of improving that intelligence. This is the one assumption I most skeptical of. In my experience, each time you make a system more clever, you also make it MUCH more complex. Maybe there is not hard limit on intelligence, but maybe each generation of improved intelligence actually takes lon…

[deleted]

Re: Superintelligence: The Idea That Eats Smart People

#209

Post by paulbaumgart » Fri, Dec 23, 2016, 1:43 AM UTC

kazagistar wrote: post #189

They always miss a critical and subtle assumption: that intelligence scales equal to or faster then the computational complexity of improving that intelligence. This is the one assumption I most skeptical of. In my experience, each time you make a system more clever, you also make it MUCH more complex. Maybe there is not hard limit on intelligence, but maybe each generation of improved intelligence actually takes lon…

Relevant discussion elsewhere in this thread: https://news.ycombinator.com/item?id=13241223

Re: Superintelligence: The Idea That Eats Smart People

#210

Post by trishume » Fri, Dec 23, 2016, 1:44 AM UTC

idlewords wrote: post #56

apsec112 wrote: post #29

"Not many people know that Einstein was a burly, muscular fellow. But if Einstein tried to get a cat in a carrier, and the cat didn't want to go, you know what would happen to Einstein. He would have to resort to a brute-force solution that has nothing to do with intelligence, and in that matchup the cat could do pretty well for itself." This seems, actually, like a perfect argument going in the other direction. Ever…

I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.

The issue is that in this case "vastly smarter" is not smart enough to truly understand the cat. It's conceivable an AI with tons of computing power could simulate a cat and reverse engineer its brain to find any stimulus that would cause it to get in the cage.

I also think this isn't a very good analogy. In this case we're talking about manipulating humans, where we already know manipulation is eminently possible.

Heck it wouldn't even need psychological manipulation. Hack a Bitcoin exchange or find another way of making money on the internet, then it can just pay someone absurd sums to do what it wants.