Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#121

Post by wyager » Thu, Dec 22, 2016, 11:59 PM UTC

> What kind of person does sincerely believing this stuff turn you into? The answer is not pretty.

This is a particularly stupid version of https://en.wikipedia.org/wiki/Appeal_to_consequences

"If you don't agree with me, you'll be associated with these people I'm lambasting!" I was surprised to see something so easily refutable used to conclude the argument; the article started out fairly strong.

> If you're persuaded by AI risk, you have to adopt an entire basket of deplorable beliefs that go with it.

Well if they're "deplorable", they must be false! QED.

Re: Superintelligence: The Idea That Eats Smart People

#122

Post by kobayashi » Thu, Dec 22, 2016, 11:59 PM UTC

I can't disagree enough. Having recently read Superintelligence, I can say that most of the quotes taken from Bostrom's work were disingenuously cherry-picked to suit this author's argument. S/he did not write in good faith. To build a straw man out of Bostrom's theses completely undercuts the purpose of this counterpoint. If you haven't yet read Superintelligence or this article, turn back now. Read Superintelligence, then this article. It'll quickly become clear to you how wrongheaded this article is.

Re: Superintelligence: The Idea That Eats Smart People

#123

Post by zevyoura » Fri, Dec 23, 2016, 12:00 AM UTC

modeless wrote: post #74

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

Re: Superintelligence: The Idea That Eats Smart People

#124

Post by panic » Fri, Dec 23, 2016, 12:02 AM UTC

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

It's a military AI that correctly interprets a command to kill a particular group of people, so effectively that its masters start thinking about the next group, and the next.

This has been done many times by human-run militaries; would AI make it worse somehow?

Groups of humans acting collectively can look a lot like an "AI" from the right perspective. Corporations focused on optimizing their profit spend a huge amount of collective intelligence to make this single number go up, often at the expense of the rest of society.

Re: Superintelligence: The Idea That Eats Smart People

#125

Post by darawk » Fri, Dec 23, 2016, 12:04 AM UTC

Retra wrote: post #104

darawk wrote: post #67

Earlier quoted context omitted.

All matter is, in principle, a battery. There are no physical laws that prevent near limitless energy storage.

There are if you want to have a functioning computer in the middle of all that energy. There are laws governing heat dispersal (speed of light in only 3 dimensions) and laws governing the collapse of information processing structures into black holes.

Sure, but none of those laws would prevent a superintelligence from generating more than enough electricity to power itself, in principle.

Re: Superintelligence: The Idea That Eats Smart People

#126

Post by kobeya » Fri, Dec 23, 2016, 12:05 AM UTC

jimmcslim wrote: post #36

md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

Any recommendations on decent pop-sci books that explore the latest understanding of the origin and mechanism of consciousness? Is anyone exploring this third way at all?

Daniel Dennett's book Consciousness Explained.

Re: Superintelligence: The Idea That Eats Smart People

#127

Post by marcosdumay » Fri, Dec 23, 2016, 12:05 AM UTC

idlewords wrote: post #50

stcredzero wrote: post #32

In 1945, as American physicists were preparing to test the atomic bomb, it occurred to someone to ask if such a test could set the atmosphere on fire...Los Alamos physicists performed the analysis and decided there was a satisfactory margin of safety. Exactly how big was this margin? Please someone tell me this was quite a large margin!

If I remember right, something like a safety factor of 30. So not really as big as people would like! More detail in this great summary: http://large.stanford.edu/courses/2015/ph241/chung1/ And the Los Alamos paper is really good reading: http://www.sciencemadness.org/lanl1_a/lib-www/la-pubs/003290...

The summary talks about a safety factor of 1000 on an extremely unreasonable scenario of 1MeV of average temperature, falling down to 10 in an incredibly more unreasonable scenario of 10MeV temperature.

I don't know the actual probabilities, but I do really expect a few atoms to get over 1MeV inside a fission bomb (as the reaction emits ~5MeV of energy). But they will almost certainly not hit each other. I also don't see how any atom anywhere can reach 10MeV.

Re: Superintelligence: The Idea That Eats Smart People

#128

Post by dragonwriter » Fri, Dec 23, 2016, 12:05 AM UTC

idlewords wrote: post #89

zoren wrote: post #76

Earlier quoted context omitted.

As to the first claim. It seems Mississippi has the highest poverty rate at 21.9%. California is at 16.4%. Source: https://en.wikipedia.org/wiki/List_of_U.S._states_by_poverty...

This is by an antiquated measure. The accepted rate now is 20.6%, first in the nation. Discussion here: http://www.forbes.com/sites/chuckdevore/2016/09/28/why-does-...

The SPM is not, by any means, the "accepted rate". Whether it's a more appropriate measure for any particular purpose is a legitimate discussion to have.

Re: Superintelligence: The Idea That Eats Smart People

#129

Post by ajamesm » Fri, Dec 23, 2016, 12:06 AM UTC

apsec112 wrote: post #118

ajamesm wrote: post #113

Earlier quoted context omitted.

The ACA could barely roll out a website without tragic failure, how have cell phones "dramatically improved" the lives of the poor? They're still subject to as much bureaucracy and denial of basic services as ever. 4G hasn't improved public transpo. I suppose the poor are no longer subject to long-distance fees during daytime.

"how have cell phones "dramatically improved" the lives of the poor?" There's tons of stuff on this, but, eg., here's a poster from USAID: https://s-media-cache-ak0.pinimg.com/originals/09/35/2d/0935...

Did you look at these metrics? They're weak as fuck. Most of them are "there's a platform or chart for this now" which is meaningless and probably intentionally decontextualized.

To be clear, it's good that people have access to the internet and all the interconnectivity that it brings. I'm not slamming the basic premise.

What I'm angry about is the opportunity cost. Telecoms & ISPs roll out bare minimums and say "hey look, now poor people can [cherry-picked thing that doesn't alleviate poverty]" when the real question is how we live in the most prosperous nation of all time, in an era when we have solved every basic necessity, and kids still experience food insecurity in American cities. Meanwhile, telecom CEOs merge back into monopolies.

Dramatic improvement in the lives of the poor would yield better metrics than a 30% increase in Haitians using mobile banking. Christ, half of that increase could be from unconnected citizens dying from endemic disease & reducing the denominator.

Re: Superintelligence: The Idea That Eats Smart People

#130

Post by palladiol » Fri, Dec 23, 2016, 12:08 AM UTC

About the we live in a computer simulation thing. When I look at the universe or subatomic particles, I see physical reality and I don't see anything that indicates a computer simulation. We just don't know the next step. Thinking we live in a computer simulation as the next physical phenomenon for the infinitely great just seems to me to display a lack of imagination rooted in our time and resembles the belief in a god. I would tend to agree with the author in that regard that some in our industry tend toward mysticism where computers are the religion.