Critical brain hypothesis: A physical theory for when the brain performs best
1–10 of 43 posts
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#2Stuart Kauffman explored this idea years ago with his NK Theory [1]
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#3"The critical brain hypothesis suggests that neural networks do their best work when connections are not too weak or too strong."
Isn't this just about as obvious as the fact that traffic flows best when traffic lights are neither always red nor always green?
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#4"The Principles of Deep Learning" paper https://arxiv.org/abs/2106.10165
has a rather rigorous (based on Quantum Field Theory (QFT) mathematical apparatus) analysis of modern deep learning with the similar insight. They suggest that the learning happens in the critical regimes. And use running couplings, renormalization group and other fancy OFT math to derive some insights in the DL field.
Here's a HN thread, by the way, https://news.ycombinator.com/item?id=31051540.
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#5"The critical brain hypothesis suggests that neural networks do their best work when connections are not too weak or too strong." Isn't this just about as obvious as the fact that traffic flows best when traffic lights are neither always red nor always green?
I had a similar issue with the article. Essentially the information content seems to boil down to "there is a state where the brain works the best". For experts there is probably a lot to learn from the technicalities of this research, but the article leaves a layman a bit cold.
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#6"The critical brain hypothesis suggests that neural networks do their best work when connections are not too weak or too strong." Isn't this just about as obvious as the fact that traffic flows best when traffic lights are neither always red nor always green?
That makes it sound like optimising. To my not very great understanding, I think it’s more like keeping things right on the edge
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#7If you look for something in a complex system, and you look hard enough, you're probably going to find it. The example of epilepsy might just be seeing certain behavior through the lens of the theory. Unfortunately, the article fails to give us any hard definition of criticality.
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#8"The critical brain hypothesis suggests that neural networks do their best work when connections are not too weak or too strong." Isn't this just about as obvious as the fact that traffic flows best when traffic lights are neither always red nor always green?
Not really. Signals have a finite power level. If you open all the lanes all the time, you'll get a very attenuated signal throughout the entire network. If some connections are stronger than others, that's when you can actually see interesting behavior.
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#9"The critical brain hypothesis suggests that neural networks do their best work when connections are not too weak or too strong." Isn't this just about as obvious as the fact that traffic flows best when traffic lights are neither always red nor always green?
To me the fact that more information is transmitted with an intermediate number of connections than with a strongly connected network wasn't immediately obvious at first glance. I guess there is a link to entropy, i.e. how surprised can you be by the information received at one end of the network given its connectivity.
Re: Critical brain hypothesis: A physical theory for when the brain performs best
#10Any idea on how this would affect learning with a spaced repetition software? Perhaps, the practice of excessive recalling with, say, Anki could essentially be detrimental to learning in some aspects? As it would make certain connections in a neural network unnaturally strong and cause saturation and overactivation in the last layer.