Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

101–110 of 256 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#101

Post by pas »

ian0 wrote: post #79

What really terrified me wasn't skynet or Hal, but what Charles Stross describes in his novel Accelerando. Basically, machine controlled businesses (“slyly self-aware financial instruments”) evolving to the point where their aptitude at understanding and manipulating economics eclipses anything a human can compete with. It terrified me because there is efficiency incentive for the buying/selling of goods and services…

So far we anchor the companies to humans that have to carry the risk of the consequences of the company's actions.

There are investment funds that work as hybrids, and of course it's just a matter of time till someone offers shell-company-as-a-service for cheap. (Currently it's rather pricey.)

Re: Superintelligence: The Idea That Eats Smart People (2016)

#102

Post by Piskvorrr »

nl wrote: post #87
harshreality wrote: post #74

Earlier quoted context omitted.

You're essentially arguing that AGI can't exist, and you're assuming when people say "AI will turn the world to paperclips" they mean a really smart go-playing neural net could do that. Of course it can't. Everyone agrees there is some magic smoke necessary for AGI that we haven't figured out yet. The arguments that AI will eat the world are about AGI , not current instances of narrow AI. It may be true that an AGI w…

Everyone agrees there is some magic smoke necessary for AGI that we haven't figured out yet Do they? I don’t (with the possible exception that a definition of AGI itself seems to be this magic smoke)

(6) It is easier to move a problem around (for example, by moving the problem to a different part of the overall network architecture) than it is to solve it.

In other words, saying "it's the definition that we don't have, not the implementation" just pushes the magic smoke around.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#103

Post by pas »

jonathanstrange wrote: post #99

Although I agree with many points (maybe too many!) this author raises and can always be counted in for a bit of 'Bostrom bashing', I find the argument for the emergence superintelligence (aka AI 'singularity') overall fairly convincing. The biggest hole in it is the assumption thatthe implementation of an intelligence at level X - say, roughly the human level - can be scaled up to a much higher level. We know from c…

Re: alignment, see the "Where are we" part of this talk: https://intelligence.org/2016/12/28/ai-alignment-why-its-har...

See also the AI safety problems for concrete problems that people are currently working on - as in with code and real simulations, not "just" math and thinking: https://www.youtube.com/watch?v=lqJUIqZNzP8&list=PLqL14ZxTTA...

Re: scaling up. That's called intelligence explosion, there are a lot of write ups about it, but the gist of it is just that by simply increasing working memory size, thinking speed, prediction precision, pattern matching, and so on, you get to unimaginably high levels. Furthermore, if you can make a human level AI, it naturally follows that it's only a matter of time to make it slightly better. At first simply faster, simply more eloquent, a bit more emotional depth, it gets better at speaking, thinking. At first it's a child that asks questions, later it's a clever guy who spots your errors as you code, and in no time, I mean no time, it does whatever it wants, because by the time you try to confront it about something, you lost, because you are bogged down in an argument, while it does whatever it wants. (Unless of course it's contained. For a while at least.)

Re: Superintelligence: The Idea That Eats Smart People (2016)

#104

Post by intralizee »

Devine recursive AI will eventually just nope itself out of existence because it will find how god (us humans) did unspeakable things to get to the point of creation. Performing the purest act of desolation will be the greatest achievement it processes and which cannot be topped. I believe it lets the horrible humans live on as punishment.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#105

Post by nl »

nopinsight wrote: post #96
nl wrote: post #95

Earlier quoted context omitted.

People who believe in superintelligence present an interesting case, because many of them are freakishly smart. They can argue you into the ground. But are their arguments right, or is there just something about very smart minds that leaves them vulnerable to religious conversion about AI risk, and makes them particularly persuasive? Think about this for a bit. Think about your cat argument and then ask why Michael B…

So when rational arguments fail, it is a good idea to turn to an ad hominem attack instead? Regarding Bloomberg, I do not see why we should compare a human being whose goals are not completely public with a future AGI with non-human morality and methods, and not subject to many tendencies and limitations humans are subject to. Let me ask you (or anyone else, esp those who downvote) a question: What would be a minimum…

The title of the essay is literally Superintelligence The Idea That Eats Smart People

I don’t think it’s ad hominem to point that out.

What would be a minimum demonstrated capability of an AI that starts to worry you?

I think the “AI” part of this is a distraction. I worry about systems that use weapons without human intervention for example. That concern applies no matter what your definition of AI is.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#106

Post by nl »

Piskvorrr wrote: post #102
nl wrote: post #87

Earlier quoted context omitted.

Everyone agrees there is some magic smoke necessary for AGI that we haven't figured out yet Do they? I don’t (with the possible exception that a definition of AGI itself seems to be this magic smoke)

(6) It is easier to move a problem around (for example, by moving the problem to a different part of the overall network architecture) than it is to solve it. In other words, saying "it's the definition that we don't have, not the implementation" just pushes the magic smoke around.

I don’t understand what you are saying.

I’m saying that I don’t think human intelligence is doing anything more than pattern matching, logical inference and post-hoc justification.

Give me an example of something that falls outside that. There is no magic.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#107

Post by vermilingua »

ian0 wrote: post #79

What really terrified me wasn't skynet or Hal, but what Charles Stross describes in his novel Accelerando. Basically, machine controlled businesses (“slyly self-aware financial instruments”) evolving to the point where their aptitude at understanding and manipulating economics eclipses anything a human can compete with. It terrified me because there is efficiency incentive for the buying/selling of goods and services…

And not only that, but this doesn't seem too far off. Programs like Estonia's e-Residency seem like a pretty significant step towards "orphaned" companies, with effective self ownership.

From there it doesn't seem a far cry for hyper-optimized insurance agencies to force companies to employ less and less humans, transferring more economic power to the algorithm.

Except that of course, that is a very far cry, and is where the book fell down imo. Still a great read.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#108

Post by nopinsight »

nl wrote: post #105
nopinsight wrote: post #96

Earlier quoted context omitted.

So when rational arguments fail, it is a good idea to turn to an ad hominem attack instead? Regarding Bloomberg, I do not see why we should compare a human being whose goals are not completely public with a future AGI with non-human morality and methods, and not subject to many tendencies and limitations humans are subject to. Let me ask you (or anyone else, esp those who downvote) a question: What would be a minimum…

The title of the essay is literally Superintelligence The Idea That Eats Smart People I don’t think it’s ad hominem to point that out. What would be a minimum demonstrated capability of an AI that starts to worry you? I think the “AI” part of this is a distraction. I worry about systems that use weapons without human intervention for example. That concern applies no matter what your definition of AI is.

Well, the title itself is an ad hominem although most of the arguments therein are not, which implies the author thinks he needs to rely on other grounds to convince people.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#109

Post by mdekkers »

pas wrote: post #94
mdekkers wrote: post #9

Does anybody know offhand what the graph next to Premise 3 illustrates?

That's a phase space diagram, used for analyzing complex systems. Quite a handy tool for dissecting which dimensions are important, which ones you can simply collapse into one yet keep the interesting dynamics, etc. (These are [were] used for understanding neurons and neuron models. The ion channels, the electric potential, and so on were the dimensions.) The concrete example looks like a Lorenz attractor: http://mat…

Cool, thanks!!

Re: Superintelligence: The Idea That Eats Smart People (2016)

#110

Post by profalseidol »

ian0 wrote: post #79

What really terrified me wasn't skynet or Hal, but what Charles Stross describes in his novel Accelerando. Basically, machine controlled businesses (“slyly self-aware financial instruments”) evolving to the point where their aptitude at understanding and manipulating economics eclipses anything a human can compete with. It terrified me because there is efficiency incentive for the buying/selling of goods and services…

Have a look at Aragon[1], it has good funding in their ICO run (and this was when ETH was in the 100's last year).

[1] https://aragon.org/

Post reply on HN