Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

11–20 of 256 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#11

Post by themodelplumber »

steerpike wrote: post #5
MBlume wrote: post #4

"The orthogonality thesis is false. Source: Rick and Morty" Why does anyone take this writeup seriously?

First time encountering a Maciej idlewords post, huh? He doesn't even take himself seriously. That also doesn't stop him from being an excellent and thoughtful writer.

An extremely observant existentialist-idealist caught up in the tech world and its intelligence-vs-humanity theorist ideas. Poor guy. This happens a lot and we get some really amazing things from such gifted thinkers, for example Larry Wall's literary thinking about code. But I can also see where the annoyance comes from. AI theorizing requires huge leaps of intuition which can immediately supersede the existential in one's own mind, because you're thinking 200 or 2000 steps ahead, seeing the crises, seeing the opportunities, and pretty soon you have a level of convergent certainty which is hard to explain step by step, because you yourself don't really think in that step by step way, but rather in leaps. But Maciej _does_ need to think that way, especially regarding far-future concepts. And now you the AI-guy have both the burden of certainty and the energy to try to prioritize group attentions toward things you can see, that Maciej can't. IMO it's an important difference in psychology. And Maciej makes Maciej-like points: Don't sacrifice current generations' humanity for progress toward a "better" or "saved" future, when there are in fact many possible outcomes (this is key--divergence where others see convergence), and a pressing need to act on identified, easily-sensed problems, right now.

The question I have is, how much of this concern on Maciej's part is "wooly" extrapolation or the same kind that Maciej is complaining about? Much of what I just read also makes intuitive leaps, albeit of a different mindset. They are leaps of human judgment, leaps to fear the nature of those who "feel" inhumane. This is where I wonder if some more nuance and back-and-forth communication would be more helpful than a presentation.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#12

Post by striking »

sans-serif wrote: post #6

MIRI pretty much demolished this here https://intelligence.org/2017/01/13/response-to-ceglowski-on...

You're sure about that?

> [after discussing alphago]

> Consider, for example, an old doctor; suppose they’ve seen twenty patients a day for 250 workdays over the course of twenty years. That works out to 100,000 patient visits, which seems to be roughly the number of people that interact with the UK’s NHS in 3.6 hours. If we train a machine learning doctor system on a year’s worth of NHS data, that would be the equivalent of fifty thousand years of medical experience, all gained over the course of a single year.

Doctoring is just like playing Go, right? Just increase the CPU cycles on it and pack more data in there, it's more or less the same.

You can make that assumption but I don't think it'd be based in fact or reality, because Go has far fewer inputs and states than treating humans does. And you don't get to test every hypothesis and then rewind either.

Most answers there are like this, making unfounded conclusions into supposedly insightful rebuttals.

Let's try another.

> If we then take into account the fact that whenever one Einstein has an insight or learns a new skill, that can be rapidly transmitted to all other nodes, the fact that these Einsteins can spin up fully-trained forks whenever they acquire new computing power, and the fact that the Einsteins can use all of humanity’s accumulated knowledge as a starting point, the server farm begins to sound rather formidable.

Or maybe they tell each other fake news so quickly they can't tell what's right and what's wrong, like we do whenever we find a more efficient way to communicate?

Anyone can make unfounded assumptions about anything; I just did it. It's up to you to decide if you care whether these assumptions are based in reality or not. But if you consider yourself "rational", I think it'd be in your best interest to care.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#13

Post by Barrin92 »

sans-serif wrote: post #6

MIRI pretty much demolished this here https://intelligence.org/2017/01/13/response-to-ceglowski-on...

I honestly don't think anything in that post 'demolishes' the criticism or even advances some sort of argument.

It's just a huge wall of text full of weird analogies which is quite typical for these 'rationalist' community posts.

People like Bostrom or Yudkowsky have one thing in common. They are not engineers and they stand to gain financially (in fact it is what pays their bills) to conjure up non-scientific pie in the sky scenarios about artificial intelligence.

In Bostrom's case this goes much further, he has given this treatment to anything including nuclear energy and related fields. Andrew Ng put it quite succinctly. Worrying about this stuff is like worrying about overpopulation on Mars, and there's maybe need for one or two people in the world to work on this.

I really wish we could stop giving so much room to this because it makes engineers as a community looks like a bunch of cultists.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#15

Post by drdeca »

goatlover wrote: post #7
jack_quack wrote: post #3

He lacks humility about minds that humans cannot comprehend

Same argument can be applied by believers to God. If we can't comprehend something, then we should be careful when talking about it. "That whereof we cannot speak, thereof we must remain silent" ~ Wittgenstein

I am likely mistaken, but my impression was that that quote was after arguing that we cannot really refer to certain things in a coherent way, and that the quote advocates that we therefore do not attempt to do so. How far off am I?

Re: Superintelligence: The Idea That Eats Smart People (2016)

#16

Post by adrianN »

striking wrote: post #12
sans-serif wrote: post #6

MIRI pretty much demolished this here https://intelligence.org/2017/01/13/response-to-ceglowski-on...

You're sure about that? > [after discussing alphago] > Consider, for example, an old doctor; suppose they’ve seen twenty patients a day for 250 workdays over the course of twenty years. That works out to 100,000 patient visits, which seems to be roughly the number of people that interact with the UK’s NHS in 3.6 hours. If we train a machine learning doctor system on a year’s worth of NHS data, that would be the equiv…

Well the assumption is that AI will be able to learn similar amounts of knowledge from access to the same amount of data as a human would. That is of course totally wrong for almost all problems for today's algorithms, but that might change in the future. Alpha Go for example improved a lot by playing against itself without outside input.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#17

Post by paulsutter »

Barrin92 wrote: post #13
sans-serif wrote: post #6

MIRI pretty much demolished this here https://intelligence.org/2017/01/13/response-to-ceglowski-on...

I honestly don't think anything in that post 'demolishes' the criticism or even advances some sort of argument. It's just a huge wall of text full of weird analogies which is quite typical for these 'rationalist' community posts. People like Bostrom or Yudkowsky have one thing in common. They are not engineers and they stand to gain financially (in fact it is what pays their bills) to conjure up non-scientific pie in…

So you agree there is room for them to work on this, yet you feel they are making engineers generally look like cultists?

Maybe you’re just being oversensitive. The hype wave on AI danger is completely over, and there’s nothing wrong with people studying the question if that’s their interest.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#19

Post by jcoffland »

> The second premise is that the brain is an ordinary configuration of matter, albeit an extraordinarily complicated one.

This is the part I doubt is true. There was a time when humans thought brains were very complex systems of microscopic clockwork because gears were the pinnacle of technology.

How do people keep mistaking pattern matching with intelligence?

Re: Superintelligence: The Idea That Eats Smart People (2016)

#20

Post by dwaltrip »

goatlover wrote: post #7
jack_quack wrote: post #3

He lacks humility about minds that humans cannot comprehend

Same argument can be applied by believers to God. If we can't comprehend something, then we should be careful when talking about it. "That whereof we cannot speak, thereof we must remain silent" ~ Wittgenstein

First we must demonstrate that the alleged deity is indeed present before we worry about understanding it.
Post reply on HN