41–50 of 112 posts
"If the human brain were so simple that we could understand it, we would be so simple that we couldn't."
A mind understanding itself may violate the second law, but a few minds understanding one mind should be theoretically possible!
Earlier quoted context omitted.
There are several of these (cognitive architectures) e.g. the ACT-R model[0]. [0] https://en.m.wikipedia.org/wiki/ACT-R
Which ones aren't BS?
https://www.oxfordscholarship.com/mobile/view/10.1093/acprof...
As a scientist to say something you don’t fully understand as impossible really pisses me off.
I'm confused. This article doesn't say anything. It makes no points and has no insight. "There's a lot of data in neuroscience?" Is that the message? An unusual number of Nautilus articles frontpage HN like this one, where there doesn't seem to be any value in the article itself. What is going on?
Exemplifies:
- Data is not information. - Information is not knowledge. - Knowledge is not understanding.
We've not even left the gate of the first tier. Both exciting and intimidating, but mostly humbling. Or should be.
What we need is a Newtonian model of the brain. A model that is incomplete and "wrong", but useful and generative. While Newtonian physics may be "wrong", is much easier to learn than quantum physics or relativity, etc. Neuroscience usually focuses on precision details, but doesn't aim to tell big picture stories. There are a few exceptions, however, like Karl Friston's free energy reduction model.
The important question to ask is: is the deep learning abstraction any good?
There's a very strong case to be made that the answer is yes: deep learning systems can perform many (of course, not all, at least not yet) tasks that involve perception (computer vision/speech recognition), motor control (the recent openai robot), language understanding (machine translation/BERT/GPT), planning (alphago/dota/the deepmind protein folding), and even some symbolic reasoning (the recent work from facebook on symbolic integration https://ai.facebook.com/blog/using-neural-networks-to-solve-...). Some of these tasks are performed at such a high level that they become commercially useful, and in some cases, surpass "human level".
So here we have a "model family" -- deep learning -- with a set of principles so simple that it can be studied with intense mathematical rigor (for example, https://arxiv.org/pdf/1904.11955.pdf or https://papers.nips.cc/paper/9030-which-algorithmic-choices-...), and that produces many of the behaviors we want out of brains (and not just behavioral: see, e.g., https://arxiv.org/abs/1805.10734: " Interestingly, recent work has shown that deep convolutional neural networks (CNNs) trained on large-scale image recognition tasks can serve as strikingly good models for predicting the responses of neurons in visual cortex to visual stimuli, suggesting that analogies between artificial and biological neural networks may be more than superficial." -- this is just one of many papers that show that even under the hood, trained deep learning systems exhibit many properties of biological neural networks).
These reasons strongly suggest (imho) that deep learning is in fact the newtonian theory of neuroscience. More strongly, no other theory comes remotely close in its simplicity and explanatory power.
Earlier quoted context omitted.
There are several of these (cognitive architectures) e.g. the ACT-R model[0]. [0] https://en.m.wikipedia.org/wiki/ACT-R
Which ones aren't BS?
If you've seen some of the high resolution videos of neural activity captures from even simple fish, the slightest motor movements activate hundreds of thousands of cells in a chaotic pattern. Neural circuitry is not neatly laid out like a silicon chip, its a forest of inter-connectivity that resists analysis even with extremely detailed visualization and data captures.
I think it’s interesting that people think we’ll be able to make something that does more in a smaller space. As if there was something other than the laws of physics preventing natural selection from testing smaller structures. Or that there’s something (other than the demands of the computation itself) constraining the architectures that were tested through natural selection.
There is. Anything that can't be reached by a small number of genetically small steps that either enhance or at least maintain fitness will never be reached.