Superintelligence: The Idea That Eats Smart People • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People

#591

Post by Nevermark » Sun, Dec 25, 2016, 5:45 AM UTC

allenz wrote: post #214

apsec112 wrote: post #88

Earlier quoted context omitted.

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill y…

I agree with you that it isn't practical right now to get rid of billionaires. However, I don't think that it's some kind of economic theorem. The reasons that socialism failed are complex, and pure capitalism failed as well (think Gilded Age), which is why everyone lives in a mixed economy. It is reductionist to say that the 1976 IMF Crisis was caused by the tax rate instead of excess spending, monetary policy, and…

The problem is not the existence of rich people. The problem is that some people are getting poorer. The two are not always linked.

In other words, inequality can be a sign of good (upward mobility, vibrant economy) as well as bad (poor people getting poorer).

Fixing the latter is important. "Fixing" the first is harmful.

Any solutions should focus on giving the average and the poor the ability to improve their situation. Reducing the number of rich should never be the goal.

Re: Superintelligence: The Idea That Eats Smart People

#592

Post by Nevermark » Sun, Dec 25, 2016, 5:55 AM UTC

simonh wrote: post #407

vosper wrote: post #262

Earlier quoted context omitted.

I think it might be great to encourage the rich to spend as much as possible. Don't the expensive cars, private jets, and first class transportation support whole networks of businesses, and provide employment?

Yes, but you also need to look at the products of those people's labor and other things that labor could be used for. Do we need more people building and crewing luxury yachts, or building and operating hospitals and sheltered accommodation? In both cases people are paid to do work, but the products of that work are very different. But in practice much of the wealth of super-wealthy people is actually either tide up…

The problem is not that their are rich people buying nice things.

The problem is when poor or middle class people are unable to improve their situation or lose ground.

The only time rich people are a problem for poor people is when rich people are able to corrupt government to tilt the playing field their way. This is a problem of corrupt politicians and lack of anti-corruption law.

I think people underestimate how many economic difficulties are not caused by economic effects, but by corrupt politicians who are permitted to stack the deck against the average person as a way to fund their campaigns or rack up post-governance favors.

Re: Superintelligence: The Idea That Eats Smart People

#593

Post by Nevermark » Sun, Dec 25, 2016, 6:00 AM UTC

nohat wrote: post #334

coldtea wrote: post #82

Earlier quoted context omitted.

> This article explicitly endorses argument ad hominem That's because it's very effective in practice. In the real world (which is not a pure game of logical reasoning only played by equals and fully intelligent beings without hidden agendas), the argument ad hominem can be a very powerful way to cut through BS, even if you can't explain why they are BS by pure reason alone. E.g. say a person A with IQ 110 talks with…

It's a reasonable heuristic for when you just don't have the time or energy, but if you are giving a 45min keynote speech on the topic I think you are expected to make the effort to judge an idea on its merits.

Exactly.

The "cultists" he is arguing against are leaders of industry and science. The discourse bar should be extremely high. Way above ad hominem disses.

Re: Superintelligence: The Idea That Eats Smart People

#594

Post by Nevermark » Sun, Dec 25, 2016, 6:19 AM UTC

linschn wrote: post #554

lern_too_spel wrote: post #551

Earlier quoted context omitted.

Literally everybody who is an expert in AI is worried about how to manage super-intelligence. The standard introductory text in AI by Russell and Norvig spends almost four pages discussing the existential risk that super-intelligence poses. The risk is so obvious that it was documented by IJ Good at Bletchley Park with Turing, and I wouldn't be surprised if it were identified even before that.

I'm an expert in the field and I'm not worried. It's an industrial risk like any other.

You haven't thought much about the risk of superintelligence if you think it is a typical risk. Is that compared to poorly designed children's toys or nuclear weapons?

I would go as far as to say, "humanity" as it is defined today, is doomed, it is just a matter of time.

The only question is: Will doom play out with a dramatic disaster or as a peaceful handoff/conversion from biologically natural humans to self-designed intelligence.

Either way, natural human beings will not remain dominant in a world where self-interested agents grow smarter in technologic instead of geologic timescales.

Re: Superintelligence: The Idea That Eats Smart People

#595

Post by Nevermark » Sun, Dec 25, 2016, 6:29 AM UTC

mcguire wrote: post #564

apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

That example is bad, but the arguments aren't quite as objectionable. " What kind of person does sincerely believing this stuff turn you into? The answer is not pretty. " I'd like to talk for a while about the outside arguments that should make you leery of becoming an AI weenie. These are the arguments about what effect AI obsession has on our industry and culture:... " ...grandiosity, megalomania, avoidance of actu…

>We had the same exact arguments used against us under communism, ...

What nonsense. None of the credible people suggesting that superintelligence has risk are spouting generic arguments that apply to communism or any previous situation.

The question is not IF humanity will be replaced but WHEN and HOW.

Clearly, in a world with superintelligence growing at a technological pace, instead of evolutionary pace, natural humanity will not remain dominant for long.

So it makes enormous sense to worry about:

* Whether that transition will be catastrophic or peaceful.

* Whether it happens suddenly in 50 years or in a managed form over the next century.

* Whether the transition includes everyone, a few people, or none of us.

Re: Superintelligence: The Idea That Eats Smart People

#596

Post by Nevermark » Sun, Dec 25, 2016, 6:41 AM UTC

Confusion wrote: post #435

apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

It's not an ad hominem argument if the personal characteristics are relevant to the topic being discussed. The personal characteristics of the people in his example are have empirically been found to be a good indicator of crankhood.

Um, no.

Hawking, Musk, et. al. are highly successful people with objectively valuable contributions who are known to be about to think deeply and solve problems that others have not.

They are as far from cranks as anyone could possibly be.

Anyone can find non-argument related reasons to suggest anyone else is crazy or a cultist, because no human is completely sane.

What someone cannot do (credibly), is claim that real experts are non-expert crazies, over appearances while completely ignoring their earned credentials.

Re: Superintelligence: The Idea That Eats Smart People

#597

Post by Nevermark » Sun, Dec 25, 2016, 6:45 AM UTC

erikpukinskis wrote: post #325

tachyonbeam wrote: post #225

Earlier quoted context omitted.

I think the issue is that once do manage to build an AI that matches human capabilities in every domain, it will be trivial to exceed human capabilities. Logic gates can switch millions of times faster than neurons can pulse. The speed of digital signal also means that artificial brains won't be size-limited by signal latency in the same way that human brains are. We will be able to scale them up, optimize the hardwa…

> it will be trivial to exceed human capabilities. Logic gates can switch millions of times faster than neurons You're equating speed with quality. There's no reason to assume that. Do you think an AI will be better at catching a fieldmouse than a falcon? Do you think the falcon is limited by speed of thought? Many forms of intelligence are limited by game theory, not raw speed. The challenge isn't extracting large q…

The previous comment is not confusing speed with quality.

The point is that once we have a machine as smart as us, simply improving its speed and resources will increase its effective intelligence.

Whether a higher/faster intelligence generates additional value in any given task is beside the point. Some tasks don't benefit from increased intelligence, but that doesn't mean being smarter doesn't come with great benefits.

Re: Superintelligence: The Idea That Eats Smart People

#598

Post by Nevermark » Sun, Dec 25, 2016, 6:59 AM UTC

jmagoon wrote: post #338

TeMPOraL wrote: post #274

Earlier quoted context omitted.

There's also another thing. AI may not need to be superhuman, it may be close-but-not-quite human and yet be more effective than us - simply because we carry a huge baggage of stuff that a mind we build won't have. Trust me, if I were to be wired directly to the Internet and had some well-defined goals, I'd be much more effective at it than any of us here - possibly any of us here combined. Because as a human, I have…

What about consciousness or intelligence implies that it would be 'pure' in the sense that you describe? Wouldn't a fully conscious being have a great deal of complexity that might render it equivalent to the roommate example? Couldn't it get offended after crawling the internet and reading that a lot of people didn't like it very much? The idea that 'intelligence' is somehow an isolatable and trainable property igno…

They same could be said of flight, but now we have machines whose sole purpose is flight.

"Pure" is a bit of an extreme word, but clearly designed things are purer to particular goals than biological systems typically are.

This is partly due to the essences of what design means and partly because machines don't have to do all the things biology does, such as maintain a metabolism that continually turns over their structural parts, reproduce, find and process their own fuel, etc.

Re: Superintelligence: The Idea That Eats Smart People

#599

Post by Nevermark » Sun, Dec 25, 2016, 7:13 AM UTC

tachyonbeam wrote: post #312

TeMPOraL wrote: post #274

Earlier quoted context omitted.

There's also another thing. AI may not need to be superhuman, it may be close-but-not-quite human and yet be more effective than us - simply because we carry a huge baggage of stuff that a mind we build won't have. Trust me, if I were to be wired directly to the Internet and had some well-defined goals, I'd be much more effective at it than any of us here - possibly any of us here combined. Because as a human, I have…

Not to mention lack of motivation and tiredness. That being said, I'm sure we can make an AI that can think much faster than a human. I also think we can build an AI that can keep more than seven items in its working memory, as humans can.

Just the fact that a machine mind will be able to interface with regular digital algorithms directly will give it huge advantage over us.

Imagine if we had calculus math modules built into our brains. Now add modules for every branch of math and physics, languages, etc. The "dumb" AI and algorithms of today will be the superintelligence mental accelerators of tomorrow.

Re: Superintelligence: The Idea That Eats Smart People

#600

Post by Nevermark » Sun, Dec 25, 2016, 7:16 AM UTC

kazagistar wrote: post #295

tachyonbeam wrote: post #225

Earlier quoted context omitted.

I think the issue is that once do manage to build an AI that matches human capabilities in every domain, it will be trivial to exceed human capabilities. Logic gates can switch millions of times faster than neurons can pulse. The speed of digital signal also means that artificial brains won't be size-limited by signal latency in the same way that human brains are. We will be able to scale them up, optimize the hardwa…

But making them linearly faster to scale them up doesn't help if the difficulty of the problems they face isn't linear. If it comes to making more clever things, I strongly doubt they are even a remotely small polynomial.

The benefits from being faster accrue more quickly than algorithmic complexity suggests.

In economics where winner often takes it all, being just a little bit faster or better can produce outsized economic benefits.