Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

251–260 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#251

Post by TeMPOraL »

nine_k wrote: post #246
EGreg wrote: post #240

Earlier quoted context omitted.

The problem arises when non-state actors wreak havoc with drones and AI with inpunity. What would we do if a drone made by a no-name manyfacturer dropped some bombs in Times Square? Who would we blame when someone uses AI to actually sow social mistrust and subvert our existing systems?

I'm pretty sure that drone-tracking equipment is mounted all over NYC, and more is being deployed as we talk. I can also imagine that a rouge drone arriving from afar and large enough to carry a bomb will be shot down by police if it can't be identified. If it does not carry a bomb, police will apologize.

I'm not buying that.

In fact, the only reason such drone attacks don't happen, or why people hadn't been casually blowing each other up for the past five decades with explosives attached to RC cars / planes is that in general, people are nice to each other. There's plenty of tools out there for dedicated people to wreck havoc in populated areas. Such people simply are very, very rare.

Re: Superintelligence: The Idea That Eats Smart People

#252

Post by rntz »

maxerickson wrote: post #248
rntz wrote: post #241

Earlier quoted context omitted.

By this measure, I would say Bostrom is not a crank. Yudkowsky is less clear. I'd say no, but I'd understand if Yudkowsky trips some folks' crank detectors.

Einstein's paper on the photoelectric effect is a bit less than 7000 words. It is part of the foundation of quantum mechanics. Superintelligence: Paths, Dangers, Strategies is in the range of 100,000 words (348 pages * roughly 300 words per page). I'm not familiar with it, but looking around it isn't even clear if it even lays out any sort of concrete theory.

1. Plenty of academics write books. 2. Comparing a paper and a book for length is obviously unfair. Bostrom has also written papers: https://en.wikipedia.org/wiki/Nick_Bostrom#Journal_articles_... 3. "Concrete theory" is vague. Is it a stand-in for "I won't accept any argument by a philosopher, only physicists need apply"?

Re: Superintelligence: The Idea That Eats Smart People

#253

Post by lucker »

JamilD wrote: post #54
md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

The roundworm (c. elegans) only has 302 neurons, and about 7,000 synapses, but is capable of social behavior, movement, and reproduction. The entire connectome has been mapped, and we understand how many of these behaviors work without having to resort to additional ontological entities like your "third way". If this complex behavior can be explained using only 302 neurons, I have no doubt that the complexity of huma…

Behavior? Yes. Consciousness? Maybe not. As far as I know, no-one has ever come up with an explanation of how matter can give rise to consciousness. Obviously consciousness is affected by changes in matter — if I take certain drugs, I feel different, etc. — but there is no explanation at all for what mechanisms may give rise to consciousness in the first place, and the very idea of the material giving rise to consciousness might actually not make sense.

Re: Superintelligence: The Idea That Eats Smart People

#254

Post by lucker »

SerLava wrote: post #60
md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

I don't understand why people have such a weird problem reconciling the brain with the mind. It IS all physical. It's also an unimaginably complex fucking shitload of tiny physical objects working incredibly quickly. If your brain was big enough to see the parts working like little gears of a clock, it would probably be planet sized, or something like that. Huge. I would EVEN say it doesn't raise any interesting phil…

I don't think there's a problem reconciling the brain with the mind, if by 'mind' one means problem-solving ability. The problem is in reconciling the brain with consciousness. There is, as far as I know, no theory that explains how consciousness can arise from matter.

Re: Superintelligence: The Idea That Eats Smart People

#255

Post by aaachilless »

Chathamization wrote: post #226
aaachilless wrote: post #100

Earlier quoted context omitted.

What if you're the first human on an island full of baby tigers? I think most AI alarmists would argue that this analogy is vastly more appropriate.

I think an analogy of a baby human on an island of tigers would be vastly more appropriate. Humans would be like the tigers - they might have a lower overall intelligence, but they are mature and self sufficient. A hyper-intelligent AI would be more akin to a baby human - it must be taught and raised first. But like the baby human on the island, it wouldn't be able to be taught by its peers, or benefit from the gener…

I find both arguments equally plausible. I think there's plenty more plausible addendums too e.g. the idea that the baby human would mature incredibly fast. I'm not sure to what extent we're all throwing speculative darts here.

Re: Superintelligence: The Idea That Eats Smart People

#256

Post by TeMPOraL »

paulbaumgart wrote: post #161
mrfusion wrote: post #91

Earlier quoted context omitted.

What's his reason? I got halfway through that and realized I didn't understand.

That even if you see diminishing returns to what you can accomplish with greater intelligence, a small advantage will still compound into a large one over the theoretically infinite lifespan of an AI.

s/theoretically/practically/, and I doubt it changes the conclusion that much (in practice).

Re: Superintelligence: The Idea That Eats Smart People

#257

Post by mahyarm »

pmyjavec wrote: post #77
apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Calfornia is mismanged to hell. The bay area has some of the worst roads in the nation with very mild weather and a wealthy tax base. It cost $8billion to make 1 or 2 miles of central subway and only €11 billion to make the worlds longest tunnel under the alps. I have the same income tax rate as I did in canada yet there isnt universal healthcare and far more economic inequality. It goes on and on. If you trippled the money base I dont know how much better it would get.

Re: Superintelligence: The Idea That Eats Smart People

#258

Post by mahyarm »

mljoe wrote: post #244
apsec112 wrote: post #88

Earlier quoted context omitted.

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill y…

I don't think an income tax that punishes people for making too much money is the right way to go about it. How about instead of punishing people for being rich, discourage the filthy rich from spending money on the frivolous. For instance, set up a luxury tax on expensive cars, private jets and jet fuel, first class transportation and primary residences and hotels that are way above the average value for an area. On…

The thing about the rich is they can hire people to make loopholes out what you just described and the financal incentive to do so.

Re: Superintelligence: The Idea That Eats Smart People

#259

Post by TeMPOraL »

jimmcslim wrote: post #12

'In particular, there's no physical law that puts a cap on intelligence at the level of human beings.' Maybe not, but there are definitely very physical laws governing everything else, that a superintelligent being's ambitions would run into. A superintelligent being isn't going to be able to build a superliminal reactionless drive if the laws of the universe say it isn't possible. More relevant, a superintelligent b…

> A superintelligent being isn't going to be able to build a superliminal reactionless drive if the laws of the universe say it isn't possible.

It isn't, but if it wants one then it will likely go and discover the actual physical constraints.

> More relevant, a superintelligent being isn't going to be able to enslave us all with robots if the laws of chemistry don't permit a quantum leap in battery chemistry.

That's what differentiates even a somewhat intelligent AI from technologies we have today. Enslaving us with battery-powered robots is but one way to reach the goal. There are many others. If we stick to robots, you have gasoline-powered robots (gasoline is a liquid battery that's still 30x as dense as our solid batteries). Then you have nuclear batteries, which we know are feasible, but don't exist because of political reasons. And then again, robots are but one way to reach a goal...

Re: Superintelligence: The Idea That Eats Smart People

#260

Post by TeMPOraL »

ewjordan wrote: post #84
jimmcslim wrote: post #12

'In particular, there's no physical law that puts a cap on intelligence at the level of human beings.' Maybe not, but there are definitely very physical laws governing everything else, that a superintelligent being's ambitions would run into. A superintelligent being isn't going to be able to build a superliminal reactionless drive if the laws of the universe say it isn't possible. More relevant, a superintelligent b…

That "quantum leap in battery chemistry" is obviously physically possible, given that animals exist...

Or even forgetting biology for a second:

- gasoline is a liquid battery with much greater energy density than our solid ones

- nuclear batteries are a thing, too

Post reply on HN