The Myth of AI – Jaron Lanier

Adding to this, imagine you were part of an ancient alien species checking in on earth every so often. Around 4.5 billion years ago, Earth was formed. 500 million years later, the first self-replicators appeared. Another half billion years (now 3.5 billion years ago) for things to settle down and create the most recent ancestor from which all life descends from. Over the next 3 billion years nothing much happens, until the Phanerozoic Eon is reached around 500 million years ago where more "interesting", larger life starts to develop. But again for another half a billion years nothing that much happens other than evolution continuing playing its game of optimizing for gene replication in more and more interesting ways. Around one hundred thousand years ago, an interesting branch of a species has been developing in ways that indicate the rise of general intelligence (their few pounds of squishy matter in their heads are pretty neat: http://www.yudkowsky.net/singularity/power/). But they don't seem to be doing all that much until about 10-20 thousand years ago. And even then, less than 10 thousand years ago, they built some interesting structures like the pyramids, but their population is still tiny and their global influence is limited. And it wasn't until about 500 years ago when things really started picking up in population growth and world influence (some mark that as the discovery of a formal scientific process to build and record more and more truths about the world that are actually true). Just around 100 years ago, this species developed air travel. In about 60 years after that, they went beyond their planet to their moon. This species at present could wipe out almost all life from the planet (if it wanted, which it doesn't) in multiple ways, to really get every last bit of life exterminated would take some doing but in a large enough time span doesn't seem impossible.

An alien species would have to be observing this planet at very frequent intervals to be able to catch any of these recent developments -- because if millions of years ago they determined nothing interesting was happening, and decided to only look in every five hundred thousand years or so, it's very likely the last time they looked our species wasn't even around.

Now you take the development of an AI that is roughly human level intelligent with a given amount of resources. If you give it the same amount of resources again, it can "breed", copy itself perfectly onto them, and now we have a "population" of two AIs whose combined intelligence should roughly match that of two twin humans. Maybe it doesn't even copy perfectly, maybe it switches some things about to see if it can create something smarter without modifying its own code directly. In any case its breeding is only restricted by its ability to breed and desire to breed in the first place and the compute and power resources it needs to run a new copy, and these can always get lower than initially. What does the long-term look like? Even if the initial resources for the first one are astronomical (e.g. all computing power in use on the planet as of 2016), I would still bet you that left to breed without any other restriction it would take the AI family far less time than 100 thousand years to reach 7 billion instances. So you're looking at many, many more AIs than non-AIs, plus each with human level intelligence that isn't distributed normally (assuming soft eugenics doesn't become widespread in non-AI populations) but is mainly all the same with perhaps increases here and there (assuming a soft takeoff), each coordinating with the same goals in mind. This situation without further specification can be either extremely good for the non-AIs or extremely bad for the non-AIs. The question of whether this will happen (and whether it happens in that exact form, personally I think hard takeoff from a single AI is likely) in the next 100 years or the next hundred thousand years (perhaps a near-extinction-level event occurs forcing our species to basically start over but with an even harder battle to survive since many low-hanging-fruit resources have been depleted) might help you determine whether to worry now about doing actions that make the extremely-good more likely than the extremely-bad, but if one thinks human-level AI can't happen at all in any amount of time, that there's something special about our squishy brains or something inherently limiting about our so-far-general intelligence such that we can't solve the engineering problem of creating another intelligence directly and must always go through breeding, the argument needs to take place at a different level.

The paperclip maximizer argument (not sure if Bostrom repeats it in his book) is yet another level of argument but it's meant for those who already buy the premise of human level AI and self-improving AI but who also think increased intelligence will naturally converge to a human-loving benevolent AI that sees how self-evidently obvious it is to love all life or whatever, that we don't have to worry about all the messy details around Friendliness because they'll just fall out of the entity naturally. No one is actually worried about Clippy tiling the solar system with himself, its probability is epsilon. But a lot of people just see this one argument out of context and infer the arguer believes it to be a relevant possibility to worry about and write off the whole group as crazy...

> And there is nothing in the physical word that we have understood so far that cannot be ultimatly expressed computationally

Except for those physical phenomena that can only be modelled with a recursive form that doesn't start with coefficients defined with infinite precision - which unfortunately includes a lot of useful things like weather prediction, and even the classical mechanics of n-body systems.

How about telling me what tomorrow's Dow close is going to be? Or the oil price four years from now? Or which parts of the universe that electron over there passed through on its way to your monitor?

I think a lot of the responses here just make the point for Lanier:

"There's a whole other problem area that has to do with neuroscience, where if we pretend we understand things before we do, we do damage to science, not just because we raise expectations and then fail to meet them repeatedly, but because we confuse generations of young scientists."

Anyone who believes that consciousness is computable by definition, because it just has to be, has a dilettante-level view of the problem. Philosophers have been arguing about this for centuries, and on the whole they're not so certain about it.

So I'll repeat - we don't have adequate models for human or even animal personality, or for emotional responsiveness, or even for the self-aware perception of qualia, which is apparently one of the key things that defines consciousness.

Big data isn't a solution to this, any more than throwing the works of Shakespeare into a database and looking for clustering statistics about word proximity and sentence structure will get you a new Julius Caesar.

The reality is there are levels of intelligent behaviour - especially self-aware creative behaviour - that are completely opaque to any modelling technique available in current CS. I believe that anyone who thinks the problem is simple and just needs moar code and a faster processor thrown at it is expressing a faith-based position of hopeful belief, not a reality-based fact.

Shakespeare is Shakespeare because the writing isn't word salad. Shakespeare is interesting because of the unusual the density and richness of the experiences that are referred to - not just in the plot, but in the details of the metaphors in each sentence.

You need to model experience before you can recreate that, and you can't do that unless you have a deep understanding of what experience is. Big data gives you slightly more focussed monkeys and more typewriters, but it's in no way a solution to the basic modelling problem.

>So their predictions are wrong, but yours are correct?

Wrong/correct? We're talking about predictions, so let's talk about probability.

In the course of 20 years is it more probable that life will look more like it does today or that AI will make us all jobless?

That is why I'm "more likely" right and Elon is "more likely" wrong.

I'll put money down on it. If, in 10 years, it only takes 1 employee per shift to run an entire McDonalds, I'll give you $1,000.

And yes, I don't believe your single factor over-fit models prove anything about global warming. I'll put money down on that too.

We live in a deterministic universe. You'll need a different definition of autonomy if you want anything to have it.

You should still accept and then offload your decisions onto someone else you think is responsible enough.