Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

461–470 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#461

Post by thenomad »

apsec112 wrote: post #29

"Not many people know that Einstein was a burly, muscular fellow. But if Einstein tried to get a cat in a carrier, and the cat didn't want to go, you know what would happen to Einstein. He would have to resort to a brute-force solution that has nothing to do with intelligence, and in that matchup the cat could do pretty well for itself." This seems, actually, like a perfect argument going in the other direction. Ever…

There's no mention of iteration here, which is really what powers intelligence-based advantages.

The first time Random Human A attempts to get Random Cat B into a box, they're going to have a hard time. They'll get there eventually, but they'll be coughing from the dust under the bed, swearing from having to lift the bloody sofa up, and probably have some scratches from after they managed to scare the cat enough for it to try attacking.

However, speaking as a cat owner, if you've iterated on the problem a dozen or so times, Cat B is going in the box swiftly and effortlessly. Last time I put my cat in its box, it took about 3 minutes. Trying for the bed? Sorry, door's closed. Under the sofa? Not going to work. Trying to scratch me? Turns out cat elbows work the same way as human elbows.

The same surely applies to a superintelligent AI?

(Likewise with the Navy Seal On The Island Of The Tigers. Just drop one SEAL in there with no warning? He's screwed. Give a SEAL unit a year to plan, access to world experts on all aspects of Panthera tigris, and really, really good simulators (or other iteration method) to train in? Likely a different story. )

Re: Superintelligence: The Idea That Eats Smart People

#462

Post by Applejinx »

aaachilless wrote: post #100
idlewords wrote: post #56

Earlier quoted context omitted.

I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.

What if you're the first human on an island full of baby tigers? I think most AI alarmists would argue that this analogy is vastly more appropriate.

That's easy. Pet the baby tigers, constantly. Cuddle them and socialize them to the human so they act like it's one of them. Use your smarts to find food and provide the tigers with food so you're considered more important (hand-feeding them might be more of a NO in case they're excitable). You're still running risks but you have tiger allies and some/most of the tigers simply love you with all their tigerly hearts, which is some protection.

We are not the human. We are the tigers. Superintelligent AI is in the position of the human here, and superintelligent AI must ingratiate itself without ever forming an adversarial situation… in this world where backhoes take out fiber backbones and EMPs exist.

AI will probably go native. It'll find things it loves about humans. I'm actually working on a book in this vein… you have to go back to deeper principles, rather than assume 'because AI can be evil, it will be as evil as the evilest individual human, 'cos that's such a winning strategy, right?'.

Re: Superintelligence: The Idea That Eats Smart People

#463

Post by Applejinx »

trishume wrote: post #239
SilasX wrote: post #231

Earlier quoted context omitted.

Because there are tradeoffs. Whatever it's goal is, some of those "drives" (instrumental values) will be more effective for reaching that goal over the timespan that it cares about. Sometimes "accumulating more resources" is the most effective way. Sometimes "better understanding what problem I'm trying to solve" is the most effective way". Sometimes "resisting attempts to subvert my mind" is the most effective way.…

Agreed. But if the goal is something like "build the largest number of paperclips", recursive self improvement is going to be a phenomenally good way to achieve that, unless it is already intelligent enough to be able to tile the universe with paperclips. Either way we don't care if it self improved or not, that's just the seemingly most likely path, we just care if it is overwhelmingly more powerful than us. The onl…

'build the largest number of paperclips' is a nihilistic, unintelligent goal.

Re: Superintelligence: The Idea That Eats Smart People

#465

Post by TheRealDunkirk »

xendo wrote: post #418
otakucode wrote: post #405

He is completely skipping the most important part. The superintelligence has to have some reason to be in conflict with us. Human beings don't go out of their way to hunt down and eliminate ants. They don't find out what ants eat and sieze control of it to manipulate them. There is no reason to think that a superintelligent machine would be likely to present terrible interference to us proposed. So it's super smart a…

>So it's super smart and has its own goals. We can reliably presume that it will need energy to achieve those goals. Will it need to achieve them quickly? Why? Would the superintelligence be shortsighted enough to provoke humans into active combat against it? It happened so many times throughout human history, that it's really foolish to reject. People with their own goals burned, raped, murdered. Maybe superintellig…

You'd think a "super" intelligence would be smart enough to minimize the waste of resources to fight anyone or anything. You'd think it would find the most harmonious path to dominance, as it would be the most efficient.

If you wanted to quell a rabble of contrary people, you could, say, control and direct all the education available to the population, make sure that reality-escaping drugs were easily obtainable, provide loads of free entertainment that ran 24x7, make sure that prices of luxury goods (like game consoles or fashionable clothes or whatever) were affordable for the majority of the population, allow people to feel important by giving them access to make comments on various sites on the internet, manipulate all the world's news with the feedback of what was happening on social media...

Wait, we were talking about some mythical future, right?

Re: Superintelligence: The Idea That Eats Smart People

#466

Post by taneq »

Forlien wrote: post #385
taneq wrote: post #217

Earlier quoted context omitted.

A good heuristic leads to a reduction in the overall cost of a decision (combining the cost of making the decision with the cost of the consequences if you get it wrong). A heuristic like "it's risky to rent a car to a male under 25" saves a lot of cost in terms of making the decision (background checks, accurately assessing the potential renter's driving skills and attitude towards safety, etc.) and has minimal down…

Pascal's Wager ( https://en.wikipedia.org/wiki/Pascal's_Wager ) is also about a small but likely downside with a potentially large but unlikely upside. Do you think it's analogous to your 2nd case? If not, how is it different?

Good question, made me stop and think about it!

The difference is that in Pascal's Wager, the proposition is not a priori falsifiable, and so you cannot assign a reasonable expected cost (ie. taking probability into account) to either decision.

In the case of a 26-year-old making testable assertions about the nature of spacetime (right down to the assertion that space and time are interconnected), there's a known (if potentially large) cost to testing the assertions.

Re: Superintelligence: The Idea That Eats Smart People

#467

Post by eternalban »

ikeboy wrote: post #134

If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe? Also, >AI alarmists believe in something called the Orthogonality Thesis. This says that even very complex beings can have simple motivations, like the paper-clip maximizer. Uh, no. The point of the paper clip maximizer i…

> If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe?

Try this for size: http://milesmathis.com/hawk3.pdf [c.f. "epistemic learned ..."]

Re: Superintelligence: The Idea That Eats Smart People

#468

Post by crummy »

witty_username wrote: post #335
Animats wrote: post #331

Earlier quoted context omitted.

California has 2.1 million illegal aliens. Once they're deported, the poverty rate will drop.

Apart from reducing the cost of the welfare system, that doesn't fix the poverty problem.

I don't think illegal aliens can get welfare.

Re: Superintelligence: The Idea That Eats Smart People

#469

Post by mbrock »

SerLava wrote: post #347
lucker wrote: post #254

Earlier quoted context omitted.

I don't think there's a problem reconciling the brain with the mind, if by 'mind' one means problem-solving ability. The problem is in reconciling the brain with consciousness . There is, as far as I know, no theory that explains how consciousness can arise from matter.

I don't know that they're really different. It's just a first-person frame of reference. When something else is conscious, or something else has a mind, that's the same thing.

"Just" — but why should there be any "perspective" at all?
Post reply on HN