Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

71–80 of 256 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#71

Post by subroutine »

'Superintelligence' synthesizes the alarmist view of AI as both dangerous and inevitable

I find the 'inevitable' part quite interesting; and agree. Say we determine that if AI surpassed human intelligence we were doomed, with 100% certainty... I think it would not slow the march toward that end-point whatsoever. People would reason there is much progress to be had up-until that singularity. And since what constitute human-level intelligence is fuzzy, or that AI could mask its true cognitive abilities (see Ex Machina), a line will inevitably be crossed. Oddly, even if it meant something bad was in store, I hope to live to see that day. How exciting.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#72

Post by jcoffland »

darawk wrote: post #58
jcoffland wrote: post #57

Earlier quoted context omitted.

Turing computable does not mean it can be computed in a reasonable amount of time with a classical computer. There are many things that are known to be computable that we will never be able to compute on a classical or even quantum computer. We do not know that the brain is not quantum equivalent or better.

We also have no reason to believe that it is. Sure, it's possible that we'll happen to discover an algorithm less efficient at computing the 'intelligence' function than our brains. We may also discover one more efficient. We can't really say much at all about it at this time.

Agreed. I think we are still a long way off but some essential discovery could be just around the corner.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#73

Post by titzer »

The cat argument is completely off base IMO.

The "AI" has an almost perfect motivating lever for humans: economics. A super-intelligent AI that wanted to manipulate humans could do a lot with the stock market to feed money to its long term interests. What long term interests might those be? More servers? More storage? More hardware? More software? More AI research? Better, more programmable chips? Eyes everywhere? Robots? Automation? More control? More insight into how literally _everything_ works? Everything humans do and like? Learn how to speak all the human languages?

Imagine there was a super intelligent AI hiding in the internet _already_. Terrifying how well what we're doing aligns with what it would want. And equally terrifying is the gradient of economic advantage which is absolutely aligned with putting all of our money into tech. Transhumanism, mind uploading. We're fucking hypnotized. And those who unwittingly contribute to more tech (probably all of us here), just play right into its hands. Our interests line up wonderfully. And most of us here are rewarded with outsized salaries and stock, free food, benefits, benefits, benefits.

Well played, AI.

/tongue-in-cheek

Re: Superintelligence: The Idea That Eats Smart People (2016)

#74

Post by harshreality »

lsc wrote: post #60
vidarh wrote: post #47

Earlier quoted context omitted.

> Complex Motivations. Maybe, but if not we are screwed. And this is the key: we need to get lucky every time. If we're wrong even once and a 'bad AI' gets created and obtains the ability to copy and modify itself, we've unleashed the most dangerous virus ever and putting it back in the box will be near impossible.

you are pretty optimistic about your AI's ability to deal with novel situations.

You're essentially arguing that AGI can't exist, and you're assuming when people say "AI will turn the world to paperclips" they mean a really smart go-playing neural net could do that. Of course it can't.

Everyone agrees there is some magic smoke necessary for AGI that we haven't figured out yet. The arguments that AI will eat the world are about AGI, not current instances of narrow AI.

It may be true that an AGI will never be created, but I don't see a reason to believe that just because nobody has figured it out yet. That philosophy is just like the physicist who, shortly before fission was discovered, didn't think splitting the atom was possible. Bohr, maybe? I don't remember.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#76

Post by re-actor »

Ace17 wrote: post #28

> Having realized that the world is not a programming problem, > AI obsessives want to make it into a programming problem, > by designing a God-like machine. We invented programming to solve problems from the real world. What do you mean by a "programming problem"?

The implication is that the set of problems programming can solve and the set of problems people are trying to solve with programming are two different sets with less overlap than a programmer might think.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#77

Post by michaelscott »

vidarh wrote: post #47
red75prime wrote: post #35

I can't help but take a stab. Argument from cats. Remembering that the cat represents humanity: $5 for a group of neurons which will make the cat sit at this point (which happens to be at the entrance into a cat carrier), then $5 for making the cat backpedal a little. Emus. There are many tactics humans can't use for various reasons (saturation nuclear bombardment, for example). It doesn't say much about intelligence…

> Complex Motivations. Maybe, but if not we are screwed. And this is the key: we need to get lucky every time. If we're wrong even once and a 'bad AI' gets created and obtains the ability to copy and modify itself, we've unleashed the most dangerous virus ever and putting it back in the box will be near impossible.

This is a question of motivation on the part of such an AI. How is the AI bad? If it's killing millions, why? It would have to have some kind of "will", or objective, it's attempting to achieve and also be intelligent enough to think through every lateral strategy any human or group of humans could ever employ, for every given scenario, ever.

Possible? I think so, but unbelievably improbable. I do think pursuing countermeasures, like Elon Musk and friends are doing with Neuralink, is a wise move since, even if we never get to a superintelligent AI, we'll still augment our own intelligence greatly.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#78

Post by pas »

jcoffland wrote: post #50
krisoft wrote: post #31

Earlier quoted context omitted.

So what is the brain if not just matter?

I agree. It's likely just matter. But we still don't know everything about matter.

The threshold is protein folding. Anything lower level than that is noise, but folding itself is important, as it can go wrong sometimes and kill you. (Prions.)

And things can be approximated, sometimes very close to the real thing. Hence the connectome. Synapses are the interconnects and neurons are tiny networks of very simple computers on their own.

And it's not "just" pattern matching, it's very very likely Turing complete. (You can do counting networks, you can do conditionals, you don't really need anything else to learn, match, execute simple programs.)

We know enough to claim that it's increasingly unlikely that we are missing any relevant physical phenomena for understanding the brain and the mind. (So no quantum woo is required.)

Sure, it that unlikeliness is not zero, and it might turn out that some kind of quantum effect is important for some very strange step in the workings of the brain, but even then it's very likely that we can classically model it with good approximation. (Because as others mentioned, energy scales, and other fundamental things constrain any process going on there.)

Re: Superintelligence: The Idea That Eats Smart People (2016)

#79

Post by ian0 »

What really terrified me wasn't skynet or Hal, but what Charles Stross describes in his novel Accelerando.

Basically, machine controlled businesses (“slyly self-aware financial instruments”) evolving to the point where their aptitude at understanding and manipulating economics eclipses anything a human can compete with.

It terrified me because there is efficiency incentive for the buying/selling of goods and services to be realtime and computer based. Not just stock markets but also e-comm, smart contracts, supplier-buyer platforms, on-demand services, financing etc. The traditional infrastructure blocks (discovery, quality control, payments, compliance etc) are rapidly disappearing.

There are some yet to be addressed, but they are not huge. For example a computer cannot incorporate a company. But it could easily interact via API to a human nominee who sets one up on their behalf. Companies are already non-human in intention but are a completely rational construct.

And it doesnt even have to be anywhere near ground level. Imagine a smart contract from a private equity firm that legally forces businesses to act in a specific way based on accountancy metrics. And a bunch of smart investor bots rapidly tracking the progress of these bot-led funds. How quickly would the system optimise and what would those optimisations look like to people on the ground?

Out of all the disaster situations with AI this one strikes me as so realistic because there is so much incentive to work towards it (as opposed to creating skynet, hal or the paperclip machine).

Re: Superintelligence: The Idea That Eats Smart People (2016)

#80

Post by mrob »

vermilingua wrote: post #64
red75prime wrote: post #35

I can't help but take a stab. Argument from cats. Remembering that the cat represents humanity: $5 for a group of neurons which will make the cat sit at this point (which happens to be at the entrance into a cat carrier), then $5 for making the cat backpedal a little. Emus. There are many tactics humans can't use for various reasons (saturation nuclear bombardment, for example). It doesn't say much about intelligence…

> $5 for a group of neurons which will make cat sit... What do you mean by this? As in, cats have demonstrably had electrodes or similar used to persuade them to sit?

Most of the arguments ignore premise 5 (time scale). I'm sure that if Stephen Hawking had a magical cat-pausing time freeze machine, unlimited budget, no deadline, every sensor known to science, and a few million similarly equipped assistants, he could have figured out how to get the cat in the cat carrier. People have no intuitive comprehension of what a microsecond feels like. The article references Yudkowsky, but doesn't mention that he wrote a story about this exact issue:

https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien...

Post reply on HN