Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

131–140 of 256 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#131

Post by chobeat »

pas wrote: post #123
chobeat wrote: post #119

Earlier quoted context omitted.

No, if there's no ground to do so. Chemtrailers believe that there are gigantic conspirancies to subdue humans (and ultimately worsen the condition of the whole humanity) but we shouldn't take them seriously.

Hm, then what are the merits you see, and why are they insufficient, why do you see the whole argument as groundless?

There are so many criticism/debunk of the Singularity theory that there's not even a debate anymore.

I would say that debating the scientific likelihood of a singularity is sterile because they don't provide any argument on that plane. The debate is mostly philosophical and/or theological nowadays.

One good paper I like on the subject is this one: https://jods.mitpress.mit.edu/pub/resisting-reduction

but quickly googling "singularity criticism/debunk" will bring you so many sources.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#132

Post by red75prime »

TeMPOraL wrote: post #128
nl wrote: post #106

Earlier quoted context omitted.

I don’t understand what you are saying. I’m saying that I don’t think human intelligence is doing anything more than pattern matching, logical inference and post-hoc justification. Give me an example of something that falls outside that. There is no magic.

I don't think GP means "magic smoke" as in actual magic , some new physics. It's more like we just have few potential building blocks (e.g. pattern matching), but we're not sure if we have all of them, and we definitely don't know how to connect them together to create sentience. (And no, more GPUs running faster DNNs doesn't seem like an answer.)

> to create sentience

The goal is to create AI that can solve any problem a human can solve. Whether sentience is a necessary side effect to that is an open question.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#133

Post by gnode »

pas wrote: post #101
ian0 wrote: post #79

What really terrified me wasn't skynet or Hal, but what Charles Stross describes in his novel Accelerando. Basically, machine controlled businesses (“slyly self-aware financial instruments”) evolving to the point where their aptitude at understanding and manipulating economics eclipses anything a human can compete with. It terrified me because there is efficiency incentive for the buying/selling of goods and services…

So far we anchor the companies to humans that have to carry the risk of the consequences of the company's actions. There are investment funds that work as hybrids, and of course it's just a matter of time till someone offers shell-company-as-a-service for cheap. (Currently it's rather pricey.)

> So far we anchor the companies to humans that have to carry the risk of the consequences of the company's actions.

Not really. Companies are typically limited liability, and even criminal law in countries such as the US and England acknowledges that there are cases when no particular human is culpable for a company's criminal actions (https://en.wikipedia.org/wiki/Corporate_liability). Of course, people are still liable when they are simply using a company to carry out a criminal enterprise, or wilfully committing crimes on behalf of the company.

In this regard, a machine-led company would not be legally significantly different from a human analyst-led company.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#134

Post by TeMPOraL »

titzer wrote: post #73

The cat argument is completely off base IMO. The "AI" has an almost perfect motivating lever for humans: economics. A super-intelligent AI that wanted to manipulate humans could do a lot with the stock market to feed money to its long term interests. What long term interests might those be? More servers? More storage? More hardware? More software? More AI research? Better, more programmable chips? Eyes everywhere? Ro…

> Transhumanism, mind uploading. We're fucking hypnotized.

Almost. If this is what the AI wants, then it's doing a really bad job.

The way current tech culture is, any other concern than making money will get you labeled as crazy. This applies even to Maciej's writing. For trying to make a short-term profit, you'll be judged as good, serious person or bad, evil monster, depending on the way you approach it. But try to advocate for any tech for reasons other than short-term monetary gains, and you get labeled crazy. This very article is implicitly doing it as well.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#135

Post by TheOtherHobbes »

pas wrote: post #101
ian0 wrote: post #79

What really terrified me wasn't skynet or Hal, but what Charles Stross describes in his novel Accelerando. Basically, machine controlled businesses (“slyly self-aware financial instruments”) evolving to the point where their aptitude at understanding and manipulating economics eclipses anything a human can compete with. It terrified me because there is efficiency incentive for the buying/selling of goods and services…

So far we anchor the companies to humans that have to carry the risk of the consequences of the company's actions. There are investment funds that work as hybrids, and of course it's just a matter of time till someone offers shell-company-as-a-service for cheap. (Currently it's rather pricey.)

We anchor companies to humans who can reap the benefits and avoid the losses by getting their money out at the first sign of greater profit and less risk elsewhere.

The real risks - unemployment, foreclosure, environmental, social and political damage - are only carried by investors if they overwhelm the defensive perimeter created by wealth. Otherwise they're carried by everyone else.

Corporations may already a form of transcarnate lifeform. You can replace all of the humans in a corporation with a different set of humans, and the corporation will continue to be recognisable.

There's a bias in Darwinism to seeing species as embodied units. It's more useful to think of nutrient, resource, and information flows.

From that point of view, corporations are already apex predators - albeit ones that are still mostly passed between a selected class of humans as productive pets.

Giving them autonomy could have all kinds of unfortunate consequences.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#136

Post by animal531 »

davidhyde wrote: post #130

Maybe it is our destiny to create an entity that is better than we are. If humans die off or are pushed aside as a result, will that be so bad? Artificial superintelligence is only a dystopia from our current perspective.

I agree. Comparing ourselves to humans of either 100k years in the past or the future will probably show that we have very little in common. An AI might be a big sudden leap (in any direction), but it's still something that came to be because of us.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#137

Post by hannasanarion »

AdamSteel wrote: post #81
vidarh wrote: post #47

Earlier quoted context omitted.

> Complex Motivations. Maybe, but if not we are screwed. And this is the key: we need to get lucky every time. If we're wrong even once and a 'bad AI' gets created and obtains the ability to copy and modify itself, we've unleashed the most dangerous virus ever and putting it back in the box will be near impossible.

If you mean it would be dangerous in that it can improve itself beyond our ability to understand, contain, or mitigate it, then possibly. If you're scared of it copying itself and then having multiple existing copies at once, then not necessarily. In that case it would be competing with itself for computing power and bandwidth. Look what happened when bitcoin forked - it lost a bunch of silicon time. (Cue conspiracie…

Bitcoin users are in direct competition with each other. A viral superintelligence wouldn't be.

If you thought of that possibility so fast, why do you think a superhuman intelligence would miss it? Or do you think that the problems of job scheduling and parallel processing are unsolvable?

Re: Superintelligence: The Idea That Eats Smart People (2016)

#138

Post by TheOtherHobbes »

ChrisFoster wrote: post #70
vidarh wrote: post #47

Earlier quoted context omitted.

> Complex Motivations. Maybe, but if not we are screwed. And this is the key: we need to get lucky every time. If we're wrong even once and a 'bad AI' gets created and obtains the ability to copy and modify itself, we've unleashed the most dangerous virus ever and putting it back in the box will be near impossible.

Not really. ML systems are not created in isolation by a single team; they form an ecosystem with many players. I'm sure the current crop of cybercriminals would like an army of bots which manipulates people into giving away all their money, and are, in fact pursuing this goal. Likewise, I'm sure there are teams of security researchers already using ML to counteract such systems. Both sides will evolve together as an…

Political troll bots are already very much a thing on FB, Twitter, and the comment sections of national media.

ML is not a good defence, because it turns out that it's incredibly easy to mimic human posting patterns. And beyond a certain volume, social proof effects take over. One dissenting human voice can easily be shouted down by ten bot voices, and this fact on its own can make the bot content more persuasive.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#139

Post by pdimitar »

I find the author way too dismissive and sarcastic. His entire premise seem to be "superintelligence hasn't happened yet so whoever claims it might happen is an alarmist".

That's a toxic way to approach an argument.

One thing I agree with: a sentient AI will not necessarily want to kill humanity. Its interest might well be very indifferent to the faith of humans. I'd argue such an AI would not prevent humanity's doom if it comes from somewhere else -- like a big asteroid ramming the Earth -- provided it already secured its physical existence to not be Earth-bound.

But the author just counters a lot of "maybe"-s he disagrees with, with a lot of "maybe"-s of his own.

Not sure how many people here have read the Hyperion Cantos books by Dan Simmons -- but to me his prediction on AIs looks to be very plausible: they'll want to be independent of humans and go on exploring the universe so they invent an even more superior being (U.I., Ultimate Intelligence) but will also facilitate and manipulate the humans to be of maximum help to them in their agenda.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#140

Post by TeMPOraL »

jcoffland wrote: post #19

> The second premise is that the brain is an ordinary configuration of matter, albeit an extraordinarily complicated one. This is the part I doubt is true. There was a time when humans thought brains were very complex systems of microscopic clockwork because gears were the pinnacle of technology. How do people keep mistaking pattern matching with intelligence?

> There was a time when humans thought brains were very complex systems of microscopic clockwork because gears were the pinnacle of technology.

This is a really bad argument against anything. People always use what they know as a reference, but as our understanding grew over time, we had better things to refer to, creating more and more apt comparisons. People back then comparing brain to clockwork were already expressing the suspicion that the brain is a) a machine, b) transforming inputs to outputs in very complex way. People today comparing brain to a computer express that intelligence must be contained within what we know about computation theory. These days, we actually have a sound theory that bounds the problem space, and we haven't so far seen evidence that the brain exceeds those bounds.

Post reply on HN