Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

241–250 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#241

Post by rntz »

maxerickson wrote: post #213
apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

Einstein didn't look like a crank though. His papers are relatively short and are coherent, he either already had a PHD in physics or was associated with an advisor (I didn't find a good timeline; he was awarded the PHD in the same year he published his 4 big papers). Cranks lack formal education and spew forth the gobbledygook in reams.

By this measure, I would say Bostrom is not a crank. Yudkowsky is less clear. I'd say no, but I'd understand if Yudkowsky trips some folks' crank detectors.

Re: Superintelligence: The Idea That Eats Smart People

#242

Post by idlewords »

hueving wrote: post #236
pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Can you explain to me why you think taking all of the wealth of 100 billionaires will help the poor? 100 billion spread over the population of California is less than $3000/person. So you can wipe out all of the billionaires and give everyone $50/week for one year, what is that going to change?

$3K per capita would be an enormous economic stimulus. Or give public schools $3K per student, and you'll see huge changes. It's a lot of money.

Re: Superintelligence: The Idea That Eats Smart People

#243

Post by dsacco »

21 wrote: post #155
to3m wrote: post #150

Earlier quoted context omitted.

What percentage of people are Navy SEALs? If I were a tiger, I'd probably think this island sounded like an excellent place for a fun adventure holiday with my tiger friends, and I'd be right to do so. Most likely outcome: good food, some exercise, and we can take some monkey heads home for our tiger cubs to play with.

Fun, until the navy seal appears. It takes only one. The same thing with AI, it takes only one, no matter how many dumber ones we entertained ourselves with before.

What's up with your Navy SEAL analogy in these comments? Do you know any SEALs? Have you ever seen a tiger up close, say 20 feet or so?

One adult tiger will kill an unarmed SEAL (and any other human being) in single combat. It would barely exert itself. It would make more sense if, when you said "it only takes one", you were referring to how many tigers could kill five or six SEALs who don't have guns. Fuck it, give your SEAL a fully automatic weapon - the odds are still not in his favor against a single tiger. Large felines have been known to kill or grievously injure humans after taking five high caliber rounds.

This is exactly what idlewords' point is in his essay. Your argument about a SEAL landing on an island of tigers and somehow flipping a weird "Planet of the SEALs" scenario on them is exactly what many (most?) AI alarmists do. These ridiculous debates get in the way of good faith discussion about the real dangerous of AI technology, which is more about rapid automation with less human skin in the game than it is about the subjugation of the human species.

This sort of unrealistic scenario is really fun to talk about, but that's all it is - fun. It's not really productive, and it conveniently appeals to our sense of ego and religious anxiety. Better to do real work and talk about the problems AI will cause in the future.

Re: Superintelligence: The Idea That Eats Smart People

#244

Post by mljoe »

apsec112 wrote: post #88
pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill y…

I don't think an income tax that punishes people for making too much money is the right way to go about it. How about instead of punishing people for being rich, discourage the filthy rich from spending money on the frivolous. For instance, set up a luxury tax on expensive cars, private jets and jet fuel, first class transportation and primary residences and hotels that are way above the average value for an area. On the other end, have tax credits (not just a tax deduction) for contributing to charitable causes, or for taking business risks that drive innovation.

Re: Superintelligence: The Idea That Eats Smart People

#245

Post by TeMPOraL »

state_less wrote: post #185
cloudhead wrote: post #166

Earlier quoted context omitted.

It allows wormholes to exist, which can connect arbitrary points in spacetime, hence time travel.

I time travelled here. What's the big deal? :) State machines can be checkpoint at any state, load up whatever time you like, is there time per se?

Oh guys you talk nonsense. I do dabble in time travel at times, so I know :).

(Why I'm here today? I'm particularly fond of this time period; it's the perfect time when humans had enough computing power to do something interesting, but before the whole industry went to shit. Remind me to tell you about the future some other time.)

Re: Superintelligence: The Idea That Eats Smart People

#246

Post by nine_k »

EGreg wrote: post #240
bhouston wrote: post #197

Earlier quoted context omitted.

We don't need ai for massive differences in military effectiveness. That is already here. The us can just destroy most country and substate actors with minimal causalities. The issue is already just difficult matters like differentiating friendlies/neutrals from enemies and not creating more enemies via collateral damage and other forms of reactions.

The problem arises when non-state actors wreak havoc with drones and AI with inpunity. What would we do if a drone made by a no-name manyfacturer dropped some bombs in Times Square? Who would we blame when someone uses AI to actually sow social mistrust and subvert our existing systems?

I'm pretty sure that drone-tracking equipment is mounted all over NYC, and more is being deployed as we talk. I can also imagine that a rouge drone arriving from afar and large enough to carry a bomb will be shot down by police if it can't be identified. If it does not carry a bomb, police will apologize.

Re: Superintelligence: The Idea That Eats Smart People

#247

Post by TeMPOraL »

idlewords wrote: post #56
apsec112 wrote: post #29

"Not many people know that Einstein was a burly, muscular fellow. But if Einstein tried to get a cat in a carrier, and the cat didn't want to go, you know what would happen to Einstein. He would have to resort to a brute-force solution that has nothing to do with intelligence, and in that matchup the cat could do pretty well for itself." This seems, actually, like a perfect argument going in the other direction. Ever…

I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.

Well, as a cat owner I give you this: like with any other animal, there are tricks you can exploit to coerce a cat without using physical force.

One way to get a cat into a carrier - well, the catfood industry created those funny little dry food pellets that are somehow super-addictive. Shake the box, my cat will come. Drop one in the carrier, it surely will enter. Will it eventually adapt to the trick? Maybe, but not likely if I also do this occasionally without closing the carrier door behind the cat.

Yes, we can outsmart the cat. Cats are funny because they do silly, unpredictable things at random, not because they can't be reliably tricked.

Re: Superintelligence: The Idea That Eats Smart People

#248

Post by maxerickson »

rntz wrote: post #241
maxerickson wrote: post #213

Earlier quoted context omitted.

Einstein didn't look like a crank though. His papers are relatively short and are coherent, he either already had a PHD in physics or was associated with an advisor (I didn't find a good timeline; he was awarded the PHD in the same year he published his 4 big papers). Cranks lack formal education and spew forth the gobbledygook in reams.

By this measure, I would say Bostrom is not a crank. Yudkowsky is less clear. I'd say no, but I'd understand if Yudkowsky trips some folks' crank detectors.

Einstein's paper on the photoelectric effect is a bit less than 7000 words.

It is part of the foundation of quantum mechanics.

Superintelligence: Paths, Dangers, Strategies is in the range of 100,000 words (348 pages * roughly 300 words per page).

I'm not familiar with it, but looking around it isn't even clear if it even lays out any sort of concrete theory.

Re: Superintelligence: The Idea That Eats Smart People

#249

Post by daveguy »

ikeboy wrote: post #134

If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe? Also, >AI alarmists believe in something called the Orthogonality Thesis. This says that even very complex beings can have simple motivations, like the paper-clip maximizer. Uh, no. The point of the paper clip maximizer i…

> If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe?

That Elon Musk, Bill Gates and Stephen Hawking etc are all a little nutters when it comes to UFOs and aliens?

I don't know a lot about UFOs and affiliated cults, but I'm going to guess that those mentioned are not UFO experts just like they aren't machine learning experts.

> That's not really a premise. A better version would be "a likely path to super-intelligence will be a self-improving agent".

Excellent point, but that doesn't give a self-improving agent the ability to ignore computational complexity or the uncertainty of chaotic systems.

Re: Superintelligence: The Idea That Eats Smart People

#250

Post by [deleted] »

daveguy wrote: post #249
ikeboy wrote: post #134

If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe? Also, >AI alarmists believe in something called the Orthogonality Thesis. This says that even very complex beings can have simple motivations, like the paper-clip maximizer. Uh, no. The point of the paper clip maximizer i…

> If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe? That Elon Musk, Bill Gates and Stephen Hawking etc are all a little nutters when it comes to UFOs and aliens? I don't know a lot about UFOs and affiliated cults, but I'm going to guess that those mentioned are not UFO e…

[deleted]
Post reply on HN