What Google DeepMind Means for A.I.

It's also a big challenge for AI safety / Machine Ethics / Formal Verification. It's notoriously hard to prove statements about Emergent behavior in complex or dynamic systems.

I think you'll need to look at the other end of the spectrum to see an abundance of (wrong?) models of causality: Religion and Law.

There are no "confirmed" cases of anyone actually going to heaven or hell or purgatory (or whatever else), and yet many of us still conform to some arbitary ruleset in the hopes of eventually ending (or not ending) up in one of thoses places, because we have constructed some model of how doing this gets you into hell and doing that gets you into heaven.

Similarly, we have plenty of evidence on how companies spend huge effort on finding loopholes in tax laws in order to avoid taxes, and yet instead of simplyfing the ruleset (so that there are obviously no holes in it) we still opt for piling on more laws (so that there are no obvious holes in it) because we construct (faulty?) models of how those new rules will prevent further exploits.

If it does, you don't need a spec of that state, since we know that it can emerge from something simpler (humans start out as a single cell, after all, and so in fact did all of humanity). You don't need the whole system, just the right initial conditions.

At that point you're growing a brain rather than engineering one, and maybe it takes you no closer to understanding the mechanics. But the point stands that it must be possible to construct a brain in principle, because it's already happened so many times before.

The article says something to the effect of "no matter how much you advance this strategy, you never get a toddler out of it." And that makes sense because, presumably, certain parts of the human brain exercise some sort of top-down control over the sensory-data-processing and other parts.

For example, it seems like the human mind is built to see things as things. Does the human mind reallY start off seeing "pixels" and then learn by itself to think of the word as solid, whole objects instead of collections of similarly-colored photons/pixels or atoms? It seems like this is a universal use-case and it would make sense if our tendency to see the world in terms of "things" instead of patches of color is built-in (gestalt psychology seems to suggest this as well).

It sounds like the AI in the article starts off from pixels and then builds up some sort of model of blocks, the ball, paddle, game physics, etc, (but then again, maybe it doesn't have those models at all and is just doing statistical analysis on patterns of pixels). Either way, it likely doesn't have any higher, context-independent model of objects/things like humans do. I suspect this may be one of the hurdles in transfer learning. Humans think of objects as having certain properties. When other objects in other contexts appear to have similar properties, we guess that they may have other properties in common which gives at least a rough model of the new object.

So I guess what I'm trying to say is: Humans have hierarchical models of the world that let us think separately about patterns of light, atoms/molecules, whole physical objects/things, systems, etc. They are all first-class citizens and we ascribe properties to each of them. We already have a rough-model of anything at the same level, but a different context, and with similar-enough properties to something we already know. It seems to me like this is fundamentally connected to humans' ability to do transfer-learning. Could this effect be achieved through bottom-up algorithms, or are we going to have to figure out some top-down way of developing transferrable, generalizable, hierarchical models?

Whilst technically correct, unguided evolution doesn't necessarily help us.

We know that intelligence can be achieved by one brain's worth of matter, suitably arranged, in a few years. In fact, with an extra 9 months and a suitable environment, we can do the same with a single fertilised egg. Yet reproducing these feats artificially is well beyond our current abilities.

On the other hand, evolution required a whole planet and billions of years before it stumbled on intelligence; many orders of mangnitude more effort than the above.