Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#311

Post by PavlovsCat » Fri, Dec 23, 2016, 4:04 AM UTC

zevyoura wrote: post #123

modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

That's like asking whether we really have to worry about global wars over resources since there are already knife stabbings and car crashes, or whether the Nazis were really that much different from having a bit of a cold. Sadly, being serious about this subject is not one of the strengths of HN it seems, I was kind of spooked when I only got 1 upvote for this: https://news.ycombinator.com/item?id=11685995

Re: Superintelligence: The Idea That Eats Smart People

#312

Post by tachyonbeam » Fri, Dec 23, 2016, 4:07 AM UTC

TeMPOraL wrote: post #274

tachyonbeam wrote: post #225

Earlier quoted context omitted.

I think the issue is that once do manage to build an AI that matches human capabilities in every domain, it will be trivial to exceed human capabilities. Logic gates can switch millions of times faster than neurons can pulse. The speed of digital signal also means that artificial brains won't be size-limited by signal latency in the same way that human brains are. We will be able to scale them up, optimize the hardwa…

There's also another thing. AI may not need to be superhuman, it may be close-but-not-quite human and yet be more effective than us - simply because we carry a huge baggage of stuff that a mind we build won't have. Trust me, if I were to be wired directly to the Internet and had some well-defined goals, I'd be much more effective at it than any of us here - possibly any of us here combined. Because as a human, I have…

Not to mention lack of motivation and tiredness. That being said, I'm sure we can make an AI that can think much faster than a human. I also think we can build an AI that can keep more than seven items in its working memory, as humans can.

Re: Superintelligence: The Idea That Eats Smart People

#313

Post by yakult » Fri, Dec 23, 2016, 4:10 AM UTC

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

There are at least two failure cases here:

- a military AI in the hands of bad actors that does bad stuff with it intentionally.

- a badly coded runaway AI that destroys earth.

These two failure modes are not mutually exclusive. When nukes were first developed, the physicists thought there is a small but plausible chance, around 1%, that detonating a nuke would ignite the air and blow up the whole world.

Let's imagine we live in a world where they're right. Let's suppose somebody comes around and says, "let's ignore the smelly and badly dressed and megalomanic physicists and their mumbo jumbo, the real problem is if a terrorist gets their hands on one of these and blows up a city."

Well, yes, that would be a problem. But the other thing is also a problem. And it would kill a lot more people.

Re: Superintelligence: The Idea That Eats Smart People

#314

Post by XFrequentist » Fri, Dec 23, 2016, 4:10 AM UTC

maxerickson wrote: post #213

apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

Einstein didn't look like a crank though. His papers are relatively short and are coherent, he either already had a PHD in physics or was associated with an advisor (I didn't find a good timeline; he was awarded the PHD in the same year he published his 4 big papers). Cranks lack formal education and spew forth the gobbledygook in reams.

He was awarded the doctorate for one of the papers (the photovoltaic one, if memory serves), after extending it one sentence to meet the minimum length requirement.

Re: Superintelligence: The Idea That Eats Smart People

#315

Post by ahh » Fri, Dec 23, 2016, 4:12 AM UTC

idlewords wrote: post #242

hueving wrote: post #236

Earlier quoted context omitted.

Can you explain to me why you think taking all of the wealth of 100 billionaires will help the poor? 100 billion spread over the population of California is less than $3000/person. So you can wipe out all of the billionaires and give everyone $50/week for one year, what is that going to change?

$3K per capita would be an enormous economic stimulus. Or give public schools $3K per student, and you'll see huge changes. It's a lot of money.

> Or give public schools $3K per student, > and you'll see huge changes. It's a lot of money.

Yes, you'll see them spend $3K more per student. You won't actually see any improvements in the students, though.

http://washington.cbslocal.com/2014/04/07/study-no-link-betw...

The public schools have many problems; lack of money is not one of them. Having any idea what to do with it is.

Re: Superintelligence: The Idea That Eats Smart People

#316

Post by ahh » Fri, Dec 23, 2016, 4:14 AM UTC

idlewords wrote: post #306

Eliezer wrote: post #288

Earlier quoted context omitted.

I'm getting kind of sick of people who haven't read 2% of the stuff imagining what they think we've never talked about over the last 16 years. https://intelligence.org/files/IEM.pdf

Reading 2% of your stuff is like reading 400% of my stuff, and I'm verbose as hell. Edit.

First off, that's a blatant ad hominem.

Moreover, if your best argument against the guy you're claiming is defrauding everyone is "I can't be bothered to read his work"...

Re: Superintelligence: The Idea That Eats Smart People

#317

Post by kobayashi » Fri, Dec 23, 2016, 4:17 AM UTC

Apocryphon wrote: post #309

kobayashi wrote: post #279

Earlier quoted context omitted.

Too late to edit, so I'll post just a few examples here: >The only way out of this mess is to design a moral fixed point, so that even through thousands and thousands of cycles of self-improvement the AI's value system remains stable, and its values are things like 'help people', 'don't kill anybody', 'listen to what people want'. Bostrom absolutely did not say that the only way to inhibit a cataclysmic future for hu…

To be honest, this article is hardly written completely with a straight face. It has a cheeky tone throughout. Which isn't to say it doesn't provide interesting points for the layman.

Of course, it does have a cheeky tone, though I think all of my points stand. The "interesting points for a layman" are actually a series of straw-man propaganda arguments. It does not argue in good faith and it should not be afforded the legitimacy of a thoughtful opposing position.

Re: Superintelligence: The Idea That Eats Smart People

#318

Post by curiousGambler » Fri, Dec 23, 2016, 4:23 AM UTC

TeMPOraL wrote: post #265

taneq wrote: post #219

Earlier quoted context omitted.

That actually is a good (in the sense of 'effective') heuristic. It's just not socially palatable in modern, Western civilization.

Seconding the point. If you want to accept "ad hominem" or stereotypes as a useful heuristic, you'll quickly hit things that will get you labeled as ${thing}ist. This is an utterly hypocritical part of our culture.

I've been thinking about this a lot lately, and am coming to the conclusion that it's similar to dead-weight loss in a taxation scenario. As a society we've accepted the "lower efficiency" and deadweight loss of rejecting {thing}ism because we don't want any one {thing} to get wrongly persecuted only on the basis of it being such a {thing}.

Re: Superintelligence: The Idea That Eats Smart People

#319

Post by erikpukinskis » Fri, Dec 23, 2016, 4:25 AM UTC

apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

Your assumption is that billions of dollars can be simply converted into poverty reduction.

It seems possible to me that the technology to turn money into less poverty not only doesn't exist, but that the social structures that make men like Bill Gates rich also make it difficult to create such technology.

Your implicit argument is that todays rich somehow care more about improving society than yesterday's did, which will cause these concentrations of wealth to lead to a different outcome. I'm not sure I see much of a difference between Gates and Carnegie. Different ideas about what the world needs, but not a particularly different approach to capital.

Re: Superintelligence: The Idea That Eats Smart People

#320

Post by rspeer » Fri, Dec 23, 2016, 4:25 AM UTC

ikeboy wrote: post #297

rspeer wrote: post #281

Earlier quoted context omitted.

> That's not really a premise. A better version would be "a likely path to super-intelligence will be a self-improving agent". If I understand your claim, it's not very relevant. You seem to be saying that, given that superintelligence happens, the probability that it will have happened via a self-improving agent is high. That doesn't refute the claim that "superintelligence probably will not happen". > You have a po…

> You seem to be saying that, given that superintelligence happens, the probability that it will have happened via a self-improving agent is high. I'm saying that there's no need to assume that "any intelligent agent will want to recursively self-improve". It's a strawman he deliberately uses so that rejecting it lets him get a dig in at his opponents. Of course not every agent will want to improve that, and nobody r…

Sure, there are also papers about theorem proving and stuff. I can't fault the GOFAI (Good Old-Fashioned AI) part of what MIRI does. I do believe we need more of that kind of research and it will let us understand current AI better.

But superintelligence and AI risk are made out (by many) to be the most important thing in the world. Basic research isn't that. It's just good. If we fund math and theoretical research, our descendants will be better cognitively equipped to deal with whatever they need to deal with, and that's pretty neat. I see the basic research MIRI does as a special case of that.

Wait a minute. Is the superintelligence hype just a front for convincing people to financially support mathematics and philosophy? Maybe the ends justify the means, then. But too much false urgency will create a backlash where nobody respects the work that was done, like Y2K.