Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

91–100 of 256 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#91

I can't help but take a stab. Argument from cats. Remembering that the cat represents humanity: $5 for a group of neurons which will make the cat sit at this point (which happens to be at the entrance into a cat carrier), then $5 for making the cat backpedal a little. Emus. There are many tactics humans can't use for various reasons (saturation nuclear bombardment, for example). It doesn't say much about intelligence…

> $5 for a group of neurons which will make cat sit... What do you mean by this? As in, cats have demonstrably had electrodes or similar used to persuade them to sit?

In the argument the cat represents humanity, which is being manipulated by a superintelligence. Humanity is not a single organism, so you can pay a group of people to do some things for you.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#92

> The penultimate premise is if we create an artificial intelligence, whether it's an emulated human brain or a de novo piece of software, it will operate at time scales that are characteristic of electronic hardware (microseconds) rather than human brains (hours). I never understood that argument, why would that be true? If I train a state of the art convnet it takes days on the most efficient hardware I can efford,…

[deleted]

Re: Superintelligence: The Idea That Eats Smart People (2016)

#93

> If we knew enough, and had the technology, we could exactly copy its structure and emulate its behavior with electronic components, just like we can simulate very basic neural anatomy today. Being able to simulate a brain with electronics, if it is possible, is not enough. You have to simulate it without a large increase in the space or time requirements. If there are 100B neurons in the human brain but it takes 10…

Not really disagreeing with your, but a couple of points:

1. Signal propagation between organic neurons is slow - about 120 m/s. An electrical simulation may have problems with density, but this might be offset by the much greater speed of electrical current.

2. Neurons are electrochemically complex and afaik we still don't know how to model their behaviour with high fidelity.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#94
post #9

Does anybody know offhand what the graph next to Premise 3 illustrates?

That's a phase space diagram, used for analyzing complex systems. Quite a handy tool for dissecting which dimensions are important, which ones you can simply collapse into one yet keep the interesting dynamics, etc. (These are [were] used for understanding neurons and neuron models. The ion channels, the electric potential, and so on were the dimensions.)

The concrete example looks like a Lorenz attractor: http://mathworld.wolfram.com/LorenzAttractor.html

https://en.wikipedia.org/wiki/Lorenz_system < many more images :)

Re: Superintelligence: The Idea That Eats Smart People (2016)

#95

> The Argument From Gilligan's Island > A recurring flaw in AI alarmism is that it treats intelligence as a property of individual minds, rather than recognizing that this capacity is distributed across our civilization and culture. This is a stronger argument in the post and we arguably did not know whether it is true when the post was made. AlphaZero destroyed the illusion in 2017. Thousands of years of human Go an…

People who believe in superintelligence present an interesting case, because many of them are freakishly smart. They can argue you into the ground. But are their arguments right, or is there just something about very smart minds that leaves them vulnerable to religious conversion about AI risk, and makes them particularly persuasive?

Think about this for a bit.

Think about your cat argument and then ask why Michael Bloomberg isn’t president.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#96
post #95

> The Argument From Gilligan's Island > A recurring flaw in AI alarmism is that it treats intelligence as a property of individual minds, rather than recognizing that this capacity is distributed across our civilization and culture. This is a stronger argument in the post and we arguably did not know whether it is true when the post was made. AlphaZero destroyed the illusion in 2017. Thousands of years of human Go an…

People who believe in superintelligence present an interesting case, because many of them are freakishly smart. They can argue you into the ground. But are their arguments right, or is there just something about very smart minds that leaves them vulnerable to religious conversion about AI risk, and makes them particularly persuasive? Think about this for a bit. Think about your cat argument and then ask why Michael B…

So when rational arguments fail, it is a good idea to turn to an ad hominem attack instead?

Regarding Bloomberg, I do not see why we should compare a human being whose goals are not completely public with a future AGI with non-human morality and methods, and not subject to many tendencies and limitations humans are subject to.

Let me ask you (or anyone else, esp those who downvote) a question:

What would be a minimum demonstrated capability of an AI that starts to worry you?

Re: Superintelligence: The Idea That Eats Smart People (2016)

#97

This idea that we’re close to some runaway superintelligence has been a theme through the history of AI. I look at AI today and I see some powerful techniques for optimization and categorization. There’s still an enormous leap we need to make before we’re making something that has the autonomous intelligence of a fly. We’re at least 20 years away from the first signs of real intelligence—the biggest problem is that w…

Not a fly, but we're getting there: https://en.wikipedia.org/wiki/OpenWorm

https://www.youtube.com/watch?v=Ejf6FwIibkE https://www.youtube.com/watch?v=r2OBznK6HkI

FlyConnectome: https://www.youtube.com/watch?v=EKaeTH7UgFA

https://v2.virtualflybrain.org/

recent openworm state: https://twitter.com/tarelli/status/959077915597172736?lang=e...

Re: Superintelligence: The Idea That Eats Smart People (2016)

#98
post #47

Earlier quoted context omitted.

> Complex Motivations. Maybe, but if not we are screwed. And this is the key: we need to get lucky every time. If we're wrong even once and a 'bad AI' gets created and obtains the ability to copy and modify itself, we've unleashed the most dangerous virus ever and putting it back in the box will be near impossible.

This is a question of motivation on the part of such an AI. How is the AI bad? If it's killing millions, why? It would have to have some kind of "will", or objective, it's attempting to achieve and also be intelligent enough to think through every lateral strategy any human or group of humans could ever employ, for every given scenario, ever. Possible? I think so, but unbelievably improbable. I do think pursuing coun…

[deleted]

Re: Superintelligence: The Idea That Eats Smart People (2016)

#99
Although I agree with many points (maybe too many!) this author raises and can always be counted in for a bit of 'Bostrom bashing', I find the argument for the emergence superintelligence (aka AI 'singularity') overall fairly convincing. The biggest hole in it is the assumption thatthe implementation of an intelligence at level X - say, roughly the human level - can be scaled up to a much higher level. We know from complexity theory that this is not necessarily true. The opposite is more likely for a complex algorithm like that.

However, there is a pressing issue that the superintelligence thought experiment also raises. A colleague of mine from AI called it the value alignment problem: How do we make sure that an AI's values are sufficiently aligned with human values?

The problem with this are the human values. As it turns out at a closer look, we cannot even agree on what structure these have and there is substantial disagreement among 'experts' about what human values are or what they should be. For example, as surprising as this may sound to some of you, there is substantial disagreement among philosophers of value whether 'better than' is transitive or not. There are good arguments for and good arguments against the transitivity of overall 'better than', and that's just the tip of the iceberg. To give another example, systems of law have been studied extensively and one might try to formalize them in input/output logics or normative systems, but any closer look at real systems of laws quickly reveals that their specification is incomplete and that there are many inconsistencies in them. These inconsistencies could be modelled, of course, but the big question is whether they are features or deficiencies. Again, you will find all kinds of positions among legal scholars.

In a nutshell, we don't even know how to adequately formalize the form of human values from a normative perspective, and even if we could, we would substantially disagree about their content. However, just describing human values cannot possibly solve the value alignment problem in a satisfying way, because humans have waged wars, committed genocide, mass killings, etc. Therefore, without a reasonable theory of normative values and their structure upon which we can somehow agree, the value alignment problem is underdetermined and cannot even be tackled.

The underlying problem is that human values aren't really aligned either, of course. As a consequence, we already now have completely different sets of values being applied and incorporated into AI software, depending on who develops it. This will become a serious ethical problem in the future that requires political solutions sooner than later. That's why the controversy about the hypothetical superintelligence is actually quite beneficial, even if the argument itself is no less shaky than Bostrom's Simulation Argument.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#100
post #83

Earlier quoted context omitted.

You're essentially arguing that AGI can't exist, and you're assuming when people say "AI will turn the world to paperclips" they mean a really smart go-playing neural net could do that. Of course it can't. Everyone agrees there is some magic smoke necessary for AGI that we haven't figured out yet. The arguments that AI will eat the world are about AGI , not current instances of narrow AI. It may be true that an AGI w…

Eh, if you are talking about something truly human-like rather than what people usually mean when they say AI these days, I think there are other limits. We're running up against limits on feature-size in our integrated circuits. If you invent an AGI that can make smarter copies of itself... it's pretty likely those same physical limits will continue to apply, meaning that each iteration will be better than the last,…

[deleted]
Post reply on HN