Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

81–90 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#81

Post by apsec112 »

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

There are also nearer-term threats (although I'd likely disagree on many specifics), but I don't see how that erases longer-term threats. One nuclear bomb being able to destroy your city now doesn't mean that ten thousand can't destroy your whole country ten years down the line.

Re: Superintelligence: The Idea That Eats Smart People

#82

Post by coldtea »

apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

>This article explicitly endorses argument ad hominem

That's because it's very effective in practice.

In the real world (which is not a pure game of logical reasoning only played by equals and fully intelligent beings without hidden agendas), the argument ad hominem can be a very powerful way to cut through BS, even if you can't explain why they are BS by pure reason alone.

E.g. say a person A with IQ 110 talks with a person B of IQ 140. The second person makes a very convincing argument for why the first person should do something for them. Logically it is faultless as far as person A can see. But if the person A knows that person B is shady, has fooled people in the past, has this or that private interest in the thing happening, etc, then he might use an "argument ad hominem" to reject B's proposal. And he would be better of for it.

The "argument ad hominem" is even more useful in another very common scenario: when we don't have time to evaluate every argument we hear, but we know some basic facts about the person making the argument. The "argument ad hominem" helps us short out potentially seedy, exploitative, etc. arguments fast.

Sure, it also gives false negatives, but empirically a lot of people have found that it gives more true negatives/positives (that is, if they want to act on something someone says, without delving into it finely, the fastest effective criterion would be to go with whether they trust the person).

This is not only because we don't have the time to fully analyze all arguments/proposals/etc we hear and need to find some shortcuts (even if they are imperfect), but also because we don't have all the details to make our decisions (even if we have a comprehensive argument from the other person, there can be tons of stuff left out that will also be needed to evaluate it).

Re: Superintelligence: The Idea That Eats Smart People

#83

Post by nickbauman »

Moore's Law (on a single CPU core) effectively ended around 2009. Our programming idioms for parallel processing are still so crude, hardly anyone can write maintainable parallelized code for anything but toy problems. Until we can restart Moore's Law again, this is all academic.

http://www.globalnerdy.com/2007/09/07/multicore-musings/

Re: Superintelligence: The Idea That Eats Smart People

#84

Post by ewjordan »

jimmcslim wrote: post #12

'In particular, there's no physical law that puts a cap on intelligence at the level of human beings.' Maybe not, but there are definitely very physical laws governing everything else, that a superintelligent being's ambitions would run into. A superintelligent being isn't going to be able to build a superliminal reactionless drive if the laws of the universe say it isn't possible. More relevant, a superintelligent b…

That "quantum leap in battery chemistry" is obviously physically possible, given that animals exist...

Re: Superintelligence: The Idea That Eats Smart People

#85

Post by 21 »

idlewords wrote: post #56
apsec112 wrote: post #29

"Not many people know that Einstein was a burly, muscular fellow. But if Einstein tried to get a cat in a carrier, and the cat didn't want to go, you know what would happen to Einstein. He would have to resort to a brute-force solution that has nothing to do with intelligence, and in that matchup the cat could do pretty well for itself." This seems, actually, like a perfect argument going in the other direction. Ever…

I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.

Of course, if you take your average city dweller on your island, he will probably die of thirst before the tigers get to him. But take an (unarmed) navy seal as your human on the island and I'm pretty sure in a couple of months he will be the Tiger King.

And Hawking would just ask his assistant to put the cat into the box. You are artificially depriving him of his actual resources to make a weak point.

Re: Superintelligence: The Idea That Eats Smart People

#86

Post by marcosdumay »

idlewords wrote: post #37
drakenot wrote: post #30

Earlier quoted context omitted.

Why would they be any more constrained than our biological brains? Humans are able to calculate plenty with a relatively small amount of mass and energy.

I mean that for a lot of things, doubling our brainpower would would only extend our predictive horizon by a tiny amount. There is a large class of problems for which you have fairly good solutions up front, but where even vast amounts of additional computing power don't help you do much better.

> I mean that for a lot of things, doubling our brainpower would would only extend our predictive horizon by a tiny amount.

Unless you come out with some algorithm that is sub-exponential for the important cases, and only gets slower for stuff you don't care about, or happen very rarely in practice.

People invent those algorithms all the time, for lots of kinds of problems.

Re: Superintelligence: The Idea That Eats Smart People

#87

Post by brotherjerky »

idlewords wrote: post #73
brotherjerky wrote: post #66

> I can't point to the part of my brain that is "good at neurosurgery", operate on it, and by repeating the procedure make myself the greatest neurosurgeon that has ever lived. Ben Carson tried that, and look what happened to him. Nice, but does it fall under political ban?

The political ban ended two days after it started.

Good to know, it was a terrible idea.

Re: Superintelligence: The Idea That Eats Smart People

#88

Post by apsec112 »

pmyjavec wrote: post #77
apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth

If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill your economy. There are all the failures of communist countries, of course, but even the UK tried this during the 60s and 70s. The government went bankrupt and had to be bailed out by the IMF. Inflation peaked at 27%, unemployment was through the roof, etc.:

https://en.wikipedia.org/wiki/1976_IMF_Crisis

https://en.wikipedia.org/wiki/Winter_of_Discontent

Re: Superintelligence: The Idea That Eats Smart People

#89

Post by idlewords »

zoren wrote: post #76
apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

As to the first claim. It seems Mississippi has the highest poverty rate at 21.9%. California is at 16.4%. Source: https://en.wikipedia.org/wiki/List_of_U.S._states_by_poverty...

This is by an antiquated measure. The accepted rate now is 20.6%, first in the nation. Discussion here:

http://www.forbes.com/sites/chuckdevore/2016/09/28/why-does-...

Re: Superintelligence: The Idea That Eats Smart People

#90

Post by AnimalMuppet »

apsec112 wrote: post #16
AnimalMuppet wrote: post #11

Earlier quoted context omitted.

Er, that wasn't inventing theories about time travel , just about time.

SR and GR explicitly allow time-travel into the future. Which isn't a fully general Time Machine, of course, but is a huge change from 19th-century physics. If SR had just been invented today, and someone who thought it was crazy and didn't know the math was writing a blog post about it, I 100% expect they'd call it "the time travel theory" or some such thing.

> SR and GR explicitly allow time-travel into the future.

I presume you're talking about time dilation here. That's... a little bit true, but not really? At a minimum, it's sure a strange way of looking at it.

Post reply on HN