Earlier quoted context omitted.
What astounds me is that the human brain doesn't even know it works (at least consciously). Your brain tells your body what to do, but your consciousness has no idea how. For me, that's what's so amazing about neural networks! If you could ask an individual neuron why their f(x) is the way it is, they probably would only be able to tell you that "that gets the result the brain wants". They're like individual computer…
I wonder why we don't have several conscious thoughts at once.
I'd also bet that the first "human-type & human-level" AI will be quire insane by human standards at least if we don't get this inner attention focusing part right from the first time. Considering that this insane AI will also get super-human pretty fast, I'm pretty scared of what it would do before it gets itself to some sort of inner equilibrium or "sanity"...