Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

261–270 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#261

Post by TeMPOraL »

darawk wrote: post #125
Retra wrote: post #104

Earlier quoted context omitted.

There are if you want to have a functioning computer in the middle of all that energy. There are laws governing heat dispersal (speed of light in only 3 dimensions) and laws governing the collapse of information processing structures into black holes.

Sure, but none of those laws would prevent a superintelligence from generating more than enough electricity to power itself, in principle.

Indeed, this seems like pointing to the speed of light as a limit in a discussion about building faster cars.

Re: Superintelligence: The Idea That Eats Smart People

#262

Post by vosper »

mljoe wrote: post #244
apsec112 wrote: post #88

Earlier quoted context omitted.

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill y…

I don't think an income tax that punishes people for making too much money is the right way to go about it. How about instead of punishing people for being rich, discourage the filthy rich from spending money on the frivolous. For instance, set up a luxury tax on expensive cars, private jets and jet fuel, first class transportation and primary residences and hotels that are way above the average value for an area. On…

I think it might be great to encourage the rich to spend as much as possible. Don't the expensive cars, private jets, and first class transportation support whole networks of businesses, and provide employment?

Re: Superintelligence: The Idea That Eats Smart People

#263

Post by TeMPOraL »

goatlover wrote: post #94
TillE wrote: post #33

Earlier quoted context omitted.

That's just vitalism: https://en.wikipedia.org/wiki/Vitalism Either everything exists in the material world, or it doesn't. There's really no way of avoiding that binary. If you think there's something else, that is by definition supernatural.

It's not at all binary. Read some more philosophy. Materialism hardly encompasses all the non-supernatural views. First of all, the world could be mathematical, informational, computational, or a simulation. It could have universals in addition to material particulars. Consciousness might not be reducible to the material, or the material could all have a conscious component. There might be other things, like societie…

There's really just one question - either it works in a way we can figure out, or it doesn't. The former is "materialism", the other (for now) is "supernatural", with the possible caveat that we may one day prove that some laws governing the universe are fundamentally not comprehensible for human minds, and all other minds humans could ever create (in a Gödel's incompleteness theorem style).

Re: Superintelligence: The Idea That Eats Smart People

#264

Post by beloch »

It's one thing to worry about AI's taking over the world someday. It's quite another matter entirely to think about current military automation of WMD deployment.

Everyone's probably seen Dr. Strangelove at some point in time. If you haven't, stop reading immediately and go watch it. You will not regret this. Those who have watched it are familiar with a contrived, hilarious, but mostly plausible, scheme by which human beings could be fooled into launching an unauthorized nuclear first strike. This is with technology from half a century ago. As you watch this movie, you will be exposed to a system with checks and safeties that can be bypassed by a determined (and insane) individual. Many humans at every step of the process could have stopped the deployment, but choose to blindly follow orders, well, like machines.

What people should be worried about today is how many humans stand between a decision made by a nuclear power's leader and launch. Humans doubt. Humans blink. Humans flinch. When all the data says nuclear missiles are inbound and it's time to retaliate, humans can still say "No.", and have[1]. If you automate humans out of the system, you wind up reducing the running length of a Dr. Strangelove remake. I suspect it would be down to under five minutes today.

Thanks to popular media, we have this strange idea that taking humans out of the equation in automated weapon systems reduces the possibility for error. Individual humans can, and do, make mistakes. This is true. However, humans fix each others' mistakes in any collaborative process. Machines, on the other hand, only amplify the mistakes of the original user. If a bad leader makes a bad decision with a highly automated nuclear arsenal at his or her disposal, how many other humans will have the chance to scrutinize that decision before machines enact it?

[1]https://en.wikipedia.org/wiki/Stanislav_Petrov

Re: Superintelligence: The Idea That Eats Smart People

#265

Post by TeMPOraL »

taneq wrote: post #219
_cbsz wrote: post #170

Earlier quoted context omitted.

> If a decision heuristic often leads to the correct conclusion, then it's a good heuristic (if you are under time constraints, which we are). So, is it a good heuristic to conclude that since crime is related to poverty and minorities tend to be poor, minorities qua minorities ought to be shunned?

That actually is a good (in the sense of 'effective') heuristic. It's just not socially palatable in modern, Western civilization.

Seconding the point. If you want to accept "ad hominem" or stereotypes as a useful heuristic, you'll quickly hit things that will get you labeled as ${thing}ist. This is an utterly hypocritical part of our culture.

Re: Superintelligence: The Idea That Eats Smart People

#266

Post by philipkglass »

maxerickson wrote: post #248
rntz wrote: post #241

Earlier quoted context omitted.

By this measure, I would say Bostrom is not a crank. Yudkowsky is less clear. I'd say no, but I'd understand if Yudkowsky trips some folks' crank detectors.

Einstein's paper on the photoelectric effect is a bit less than 7000 words. It is part of the foundation of quantum mechanics. Superintelligence: Paths, Dangers, Strategies is in the range of 100,000 words (348 pages * roughly 300 words per page). I'm not familiar with it, but looking around it isn't even clear if it even lays out any sort of concrete theory.

I read Superintelligence and found it "watery" -- weak arguments mixed with sort of interesting ones, plus very wordy.

At the risk of misrepresenting the book, since I don't have it in front of me, here's what bothered me most: stating early that AI is basically an effort to approximate an optimal Bayesian agent, then much later showing that a Bayesian approach permits AI to scope-creep any human request into a mandate to run amok and convert the visible universe into computronium. That doesn't demonstrate that I should be scared of AI running amok. It demonstrates that the first assumption -- we should Bayes all the things! -- is a bad one.

If that's all I was supposed to learn from all the running-amok examples, who's the warning aimed at? AFAICT the leading academic and industry research in AI/ML isn't pursuing the open-ended-Bayesian approach in the first place, largely isn't pursuing "strong" AI at all. Non-experts are, for other reasons, also in no danger of accidentally making AI that takes over the world.

Re: Superintelligence: The Idea That Eats Smart People

#267

Post by TeMPOraL »

md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

It doesn't seem like a false dichotomy. It's about whether you believe there are things that are fundamentally not comprehensible in the universe. Things that are magic in the sense of "fuck it, that doesn't make sense, let's go get drunk instead". If you believe that all phenomena can in principle be understood given enough time and effort (including even building a better intelligence to figure it out), then you have materialistic belief.

Re: Superintelligence: The Idea That Eats Smart People

#270

Post by Spooky23 »

singlow wrote: post #131
zevyoura wrote: post #123

Earlier quoted context omitted.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.

No there isn't. It's easy enough to manipulate the individual to do anything.

We speak of Nazis frequnetly, but consider what Curtis LeMay did in war and prepared to do after the war.

AI and robots give plausible deniability and reduce the number of witnesses. They also make suicide raids more practical.

Post reply on HN