Earlier quoted context omitted.
I'm as bullish on AGI as anyone in the medium term, but deep learning is not even playing the same game as AGI, let alone in the same ballpark or having the potential to achieve it. Deep learning is still mere perception. It doesn't handle memory or processing, it just transforms input into output, typically trained by Big Data, way bigger than necessary statistically speaking, given the world we live in. AGI require…
>Deep learning is still mere perception. It doesn't handle memory or processing, There's more going on than convolutional neural nets. Architectures with memory and attention mechanisms do exist. As examples: https://arxiv.org/abs/1506.03134 https://deepmind.com/blog/differentiable-neural-computers/
What really doesn't exist is any meaningful stab at unsupervised (or self-supervised) training on completely unstructured inputs or any sort of knowledge condensation/compression, at least for time dependent problems. These are of paramount importance to the way we think, and to what we can do.
There's a lot of trivially low hanging fruit, too - I still have yet to see even a grad school thesis that starts with an N+M node recurrent network and trains an N node subnetwork to match the outputs based on fuzzed ins, and then backs that out into an unsupervised learning rule that's applicable to multiple problems. Or better, a layered network that is recurrent but striated, that tries to push weights towards the lower layers while reproducing the same outputs (hell, even with a FF network this would be an interesting problem to solve if it was unsupervised). These are straightforward problems that would open up new avenues of research if good methods were found, but are mostly unexplored right now.
I could be wrong, if I had real confidence that we were close I'd be working on this stuff, but I'm collecting a paycheck doing web dev instead...