Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

431–440 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#431

Post by kodfodrasz »

alexbecker wrote: post #424
kodfodrasz wrote: post #422

Earlier quoted context omitted.

In such case what stops the masses to create a parallel economy of their own?

I imagine that would happen, but as soon as the size of their economy was large enough to attract the notice of the Hyperclass, or even just one member, it would be completely undercut by them. I don't know what an equilibrium state would look like.

I beleive that technology would still drip, if not flow from the elite world to the outsiders, and eventually they yould maybe leave the planet, for example, and leave the elites in their world, something like in Asimov's tales where the solarians and auroreans lived comfortably with their robotic servants and autofactories happily ever after, until extinction.

Re: Superintelligence: The Idea That Eats Smart People

#432

Post by spiderfarmer »

vixen99 wrote: post #421
spiderfarmer wrote: post #394

Earlier quoted context omitted.

All the more reason for leveling the playing field.

And that reason is . . .?

Companies and individuals that manage to game the tax system should be subject to an individual tax that also works retroactively, so they don't have an advantage over companies and individuals that went with the system instead of against it. Taxes could be much lower, if only everyone paid his dues.

Re: Superintelligence: The Idea That Eats Smart People

#433

Post by grzm »

erikbye wrote: post #425
maxerickson wrote: post #248

Earlier quoted context omitted.

Einstein's paper on the photoelectric effect is a bit less than 7000 words. It is part of the foundation of quantum mechanics. Superintelligence: Paths, Dangers, Strategies is in the range of 100,000 words (348 pages * roughly 300 words per page). I'm not familiar with it, but looking around it isn't even clear if it even lays out any sort of concrete theory.

Bostrom's book is wordy, boring, filled with weak parables, analogies, lacking concretization of a theory. Like always, alarmist material sells better than realism.

lacking concretization of a theory.

Would you elaborate? It's got some pretty big names recommending it. And Bostrom himself is an Oxford University professor.

Re: Superintelligence: The Idea That Eats Smart People

#434

Post by rocqua »

zevyoura wrote: post #123
modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

There isn't a whole lot of deniability. Current drones strikes involve explosions and craters.

Re: Superintelligence: The Idea That Eats Smart People

#435

Post by Confusion »

apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

It's not an ad hominem argument if the personal characteristics are relevant to the topic being discussed. The personal characteristics of the people in his example are have empirically been found to be a good indicator of crankhood.

Re: Superintelligence: The Idea That Eats Smart People

#436

Post by pron »

This is what I think about an imminent AI danger and those who believe in it:

1. It’s not as near as you think. Current machine “intelligence” is hardly on par with the abilities of an insect, let alone anything we’d call intelligent. Yeah, it’s capable of analyzing a lot of data, but computers could always do that. We don’t really know how to move forward, and there has been little progress in theory in a long while. We haven’t made any significant theoretical breakthrough in 20-40 years.

2. Intelligence isn’t a general algorithm to solve everything. For example, humans are not good at approximating solutions to NP-complete problems. Other algorithms make a better use of computational resources to solve problems that intelligence is not good at, and super-intelligence is not required to come up with those algorithms, as they use brute force on cheap computing nodes. Intelligence also isn't necessarily good at solving human problems, many of which require persuasion through inspiration or other means.

3. We don’t know what intelligence is, so we don’t know if the intelligence algorithm (or a class of algorithms) can be improved at all. Simply running at higher speeds is no guarantee of more capabilities. For all we know, an intelligent mind operating at higher speed may merely experience time as moving very slow, and grow insane with boredom. Also, we don’t know whether the intelligence algorithm can multitask well to exploit that extra time.

But most interesting at all is what I think is at the core of the issue:

4. Intelligence is not that dangerous — or, rather, it’s not so much more dangerous than non-intelligent things. This is related to point 2 above. We can obviously see that in nature, but also in human society. Power correlates weakly with intelligence beyond some rather average point. Charisma and other character traits seem to have a much bigger impact on power. Hitler wasn’t a genius. But some smart people — because they smarter than average — fantasize about a world where intelligence is everything, and not in a binary way, but in a way that gives higher intelligence non-linearly more power. This is a power fantasy, where the advantage they possess translates to the power they lack.

Re: Superintelligence: The Idea That Eats Smart People

#437

Post by Nyubis »

erikbye wrote: post #427
Nyubis wrote: post #423

Earlier quoted context omitted.

>Have you ever programmed any piece of software that suddenly implemented features of its own? I'm not sure what particular fear mongers you're talking about, but Bostrom and the like are talking about software that's specifically made to change and improve itself on a fundamental level.

I'm saying that's the pipe dream. Software won't change on its own.

We already have software that can change on its own. Neural nets are software too, and with things like backpropagation they can update their weights, essentially changing themselves.

I'm not saying that this level of change is enough to get the disaster scenarios that Bostrom talks about, but it's a folly to say that self-changing software can't possibly exist.

Re: Superintelligence: The Idea That Eats Smart People

#438

Post by scandox »

apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

The argument ad hominem here actually refers to the credibility of the source of an argument. If someone has a clear bias (cults like money and power), then you keep in mind that their arguments are the fruit of a poisoned tree.

Re: Superintelligence: The Idea That Eats Smart People

#439

Post by Confusion »

JamilD wrote: post #54
md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

The roundworm (c. elegans) only has 302 neurons, and about 7,000 synapses, but is capable of social behavior, movement, and reproduction. The entire connectome has been mapped, and we understand how many of these behaviors work without having to resort to additional ontological entities like your "third way". If this complex behavior can be explained using only 302 neurons, I have no doubt that the complexity of huma…

Yet we can't simulate one that behaves like a real one, for any value of 'like'.

Re: Superintelligence: The Idea That Eats Smart People

#440

Post by mokash »

otakucode wrote: post #416
modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Human beings have been extremely easy to kill for our entire existence. No system of laws can possibly keep you alive if your neighbors are willing to kill you, and nothing can make them actually unable to kill you. Your neighbor could walk over and put a blade in your jugular, you're dead. They could drive into you at 15MPH with their car, you're dead. They could set your house on fire while you're asleep, you're de…

So ideally there would be an AI criminal justice system, just to balance things out.
Post reply on HN