Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

41–50 of 256 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#41

Earlier quoted context omitted.

Same argument can be applied by believers to God. If we can't comprehend something, then we should be careful when talking about it. "That whereof we cannot speak, thereof we must remain silent" ~ Wittgenstein

First we must demonstrate that the alleged deity is indeed present before we worry about understanding it.

In context of this discussion, that would be future Superintelligence.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#42
> The Argument From Gilligan's Island

> A recurring flaw in AI alarmism is that it treats intelligence as a property of individual minds, rather than recognizing that this capacity is distributed across our civilization and culture.

This is a stronger argument in the post and we arguably did not know whether it is true when the post was made. AlphaZero destroyed the illusion in 2017. Thousands of years of human Go and Chess knowledge, played and accumulated in communities of many million players, were surpassed within hours of training.

AlphaZero is limited in its domain of application but there is no guarantee that all future AIs would be similarly limited.

> The Argument From Stephen Hawking's Cat ...

> But ultimately, if the cat doesn't want to get in the carrier, there's nothing Hawking can do about it despite his overpowering advantage in intelligence.

> Even if he devoted his career to feline motivation and behavior, rather than theoretical physics, he still couldn't talk the cat into it.

An average human can set up a cat trap, and lure the cat into it. They don’t even need to do it themselves; asking or paying someone else to do it is simple enough.

Humans don’t often fall into a similarly “obvious” trap but we fall for more subtle traps all the time: popularity, money, attraction, etc. A disembodied AI can use social engineering or financial incentives to get other humans to lure their targets into such a trap.

> The Argument From My Roommate...

> It's perfectly possible an AI won't do much of anything, except use its powers of hyperpersuasion to get us to bring it brownies.

Yes, an AI can decide to lie around and just try to get brownies but we know that some kinds of AGI are potentially very powerful for achieving real-world tasks and there are groups who try hard to develop them.

This is like saying an ancient bacteria we found from ten-thousand-year-old ice melted by global warming, or a group of advanced aliens who are arriving in three decades, could be totally harmless; just rest and don’t pay much attention to them.

Most arguments I have read in the post are weak, so I’ll stop here.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#43

This idea that we’re close to some runaway superintelligence has been a theme through the history of AI. I look at AI today and I see some powerful techniques for optimization and categorization. There’s still an enormous leap we need to make before we’re making something that has the autonomous intelligence of a fly. We’re at least 20 years away from the first signs of real intelligence—the biggest problem is that w…

> There’s still an enormous leap we need to make before we’re making something that has the autonomous intelligence of a fly.

I mostly agree with your broader point, but why do you believe this? It seems to me we've built machines substantially more intelligent than flies, but it's a hard thing to measure.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#44

Earlier quoted context omitted.

how do you know? some things that were thought of as inferences in the past have been shown to be clever pattern matching.

Can you provide an example?

I believe chess prodigies were supposed to derive their superiority by inference but studies have shown that a large portion of that superiority was due to superior pattern matching.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#45

> The second premise is that the brain is an ordinary configuration of matter, albeit an extraordinarily complicated one. This is the part I doubt is true. There was a time when humans thought brains were very complex systems of microscopic clockwork because gears were the pinnacle of technology. How do people keep mistaking pattern matching with intelligence?

Because we have things like Quantum Field Theory that tells us the energy scales it's valid for, and it's experimentally checked to be valid for those scales. Since the brain doesn't presumably contain supernovas, black holes or particle accelerators, that means that we have a basic theory covering the mechanics down.

One thing we do know is that we don't yet fully comprehend how the brain works. It's a little early to say it contains nothing we cannot simulate. Quantum interactions do not require high energy physics.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#46
post #15

Earlier quoted context omitted.

Same argument can be applied by believers to God. If we can't comprehend something, then we should be careful when talking about it. "That whereof we cannot speak, thereof we must remain silent" ~ Wittgenstein

I am likely mistaken, but my impression was that that quote was after arguing that we cannot really refer to certain things in a coherent way, and that the quote advocates that we therefore do not attempt to do so. How far off am I?

My understanding is that Wittgenstein likely had metaphysics and certain philosophical problems in mind. I don't know what he would have to say concerning speculation about super AIs and their potential capabilities.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#47

I can't help but take a stab. Argument from cats. Remembering that the cat represents humanity: $5 for a group of neurons which will make the cat sit at this point (which happens to be at the entrance into a cat carrier), then $5 for making the cat backpedal a little. Emus. There are many tactics humans can't use for various reasons (saturation nuclear bombardment, for example). It doesn't say much about intelligence…

> Complex Motivations. Maybe, but if not we are screwed.

And this is the key: we need to get lucky every time.

If we're wrong even once and a 'bad AI' gets created and obtains the ability to copy and modify itself, we've unleashed the most dangerous virus ever and putting it back in the box will be near impossible.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#48
post #39

> If we knew enough, and had the technology, we could exactly copy its structure and emulate its behavior with electronic components, just like we can simulate very basic neural anatomy today. Being able to simulate a brain with electronics, if it is possible, is not enough. You have to simulate it without a large increase in the space or time requirements. If there are 100B neurons in the human brain but it takes 10…

> Just because we can easily imagine it does not make it easily achievable or even possible. It is without question possible, because we exist. It is also possible to do it quickly, because again, we exist. The only thing that is unclear is: 1. How hard it is to do artificially. 2. How far from optimum the human mind is, and how complex it is to improve upon it.

But is it possible with the technology we have discovered so far?

> It is without question possible, because we exist. It is also possible to do it quickly, because again, we exist.

It's apparent that we don't know everything.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#49
post #33

> But there's also the risk of a runaway reaction, where a machine intelligence reaches and exceeds human levels of intelligence in a very short span of time. This always puzzles me. I don't have enough knowledge about AI to be objective about that kind of statement. But deep down, I feel skeptic about it. Not long ago I saw an episode of a show occuring in the late 80's. This kid had just received a computer as a gi…

Before concluding that multitudes of researchers spent 3 decades and learned nothing of note it might be worthwhile to at least spent 5 minutes doing a search.

Imagine if you knew nothing about car safety features but never having been in an accident it didn't seem like cars are any safer or more dangerous than in the 60s.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#50
post #31

> The second premise is that the brain is an ordinary configuration of matter, albeit an extraordinarily complicated one. This is the part I doubt is true. There was a time when humans thought brains were very complex systems of microscopic clockwork because gears were the pinnacle of technology. How do people keep mistaking pattern matching with intelligence?

So what is the brain if not just matter?

I agree. It's likely just matter. But we still don't know everything about matter.
Post reply on HN