Yann LeCun's comment on AlphaGo and true AI

Wouldn't that be a self-supervised learning? Yes, as the brain lears, it gets rewarded or not, but not by any external organism - rather by itself (or a part of itself). The "learning instruction manual" is in there somewhere.

So if you are able to code an AI so that it can run experiments over itself and gradually build this "learning instruction manual" eventually it will become able to do things you never though of in the first place.

I feel that your analogy does not disprove OP's claim. You would first have to prove that the analogy is actually applicable. Theoretically, one could argue that motion is substantially different from intelligence such that the analogy does not hold.

But I am with you in that engineering / CS / ... should not wait for neuroscience to make further discoveries but continue the journey.

I could not follow your argument. Could you elaborate? (Honest question)

> About recurrent NNs?

yes, obviously, as that's the topic, but also the the rest I of what I mentioned, neuroscience, maths, leaning on logic and philosophy.

> I remember similar arguments in the early 90s

That's why I mention neuroscience, the ideas are much older.

> even when a computer is super-human in every conceivable way, there'll be arguments over whether it is 'real AI'

Of course. Just because it's superhuman, we humans wouldn't know what it is, whether it is what we think it is and if that's all there could be.

Real (from res (matter (from rehis (good as in the goods))) + ~alis (adjective suffix (from all?))) means worthy and obviously an AI is only as good as its contestants are bad. It won't be real for long before it's thrown to the trash once a better AI is found.

That'll stop when the AI can settle the argument convincingly. That's what's going on, evangelizing. And we do need that, because if not for the sake of the art itself, then as proof for the application of answers and insights other fields.

> And nobody will define their terms then either. Ultimately the discussion will be as irrelevant then as it is now

LeCun sure went along a lot further since then, and he defines the terms in software. As I said, the discussion is just about what to make of it. Of course many come up basically empty, that's why the discussion is important, and that's why I asked what do you know about it. I think it's a very basic question and not easy to grow tired of. If you work that, maybe that's different and specialized to computation.

There might not be much to say about it, all the easier then to summarize in a short post. Or there's indeed more to it, then I'd appreciate a hint, to test my own understanding and learn.

I don't really know, what LeCun talks about, or the techniques you studied, so I'm saying it. Just for perspective. I'm just generally interested in learning and computation is just one relevant and informative angle. Maybe that's what bothers you, learning to learn, and that's why its freshman bothering with it, but learning to learn is maybe really just learning, or impossible. That's the kind of logical puzzle that's to be taken half joking. Don't beat yourself up over it.

(1) Adversarial learning is unsupervised and works great. Most of language modeling is unsupervised (you predict next word, but it's not real supervision because it's self-supervision). There're many works in computer vision which are unsupervised and still give more or less reasonable performance. See f.e. http://arxiv.org/pdf/1511.05045v2.pdf for unsupervised learning in action recognition, also http://arxiv.org/pdf/1511.06434v2.pdf and http://www.arxiv-sanity.com/search?q=unsupervised

(2) ImageNet supervision gives you much information to solve other computer vision tasks. So perhaps we don't need to learn everything in unsupervised manner, we might learn most features relevant for most tasks using several supervision tasks. It is kind of cheating but very reasonable one.

Moreover,

(3) We observe now just fantastic decrease of perplexity (btw, it's all unsupervised = self-supervised). It's quite probable that in the very near future neural chat bots write reasonable stories, answer intelligibly with common sense, discuss things. All of this would be just a mere consequence of low enough perplexity. If neural net says smth inconsistent it means that it gives too much probability to some inappropriate words i.e, it's perplexity isn't optimized yet.

(4) It's quite probable that it would open a finish line for human-level AI. AI would be able to learn from textbooks, scientific articles, video lectures. Btw, http://arxiv.org/pdf/1602.03218.pdf gives a potential to synthesize IBM Watson with deep learning. May be, the finish line to human level AI has been opened already.