Who finally cares how exactly an ANN matches a human brain? Is such ANN smarter than ChatGPT?
It is more useful to use AI to develop more ecologically valid measurement methods for biology.
How deep is the brain? The shallow brain hypothesis
171–180 of 183 posts
Re: How deep is the brain? The shallow brain hypothesis
#172Earlier quoted context omitted.
Many organisms have just a handful of neurons yet exhibit complex behavior that would be impossible given the weighted connections model. Not to mention single-celled organisms that exhibit ability to navigate. The model can be the closest working model but that doesn't mean it is complete. It's very likely that cells can store memories/information independent from weights.
We can’t do that not because our mathematical neurons are too simple. We can’t do that because we don’t know the algorithms those biological neurons are running. Do you see the difference?
There is of course a difference between the two things you say. They're both the reason we can't recreate the brain in software though.
Re: How deep is the brain? The shallow brain hypothesis
#173I seem to remember research stating that an individual neuron has very complex behaviour that requires several ML “neurons” / nodes to simulate. So if you do a comparison, perhaps the brain is deeper than you’d think by just looking at the graph of neurons and their synapses. Could we construct a neutral net from nodes with more complex behaviour? Probably, but in computing we’ve generally found that it’s best to bui…
The brain backprops??????
there is no evidence to support this
Re: How deep is the brain? The shallow brain hypothesis
#174I seem to remember research stating that an individual neuron has very complex behaviour that requires several ML “neurons” / nodes to simulate. So if you do a comparison, perhaps the brain is deeper than you’d think by just looking at the graph of neurons and their synapses. Could we construct a neutral net from nodes with more complex behaviour? Probably, but in computing we’ve generally found that it’s best to bui…
Yeah, biological brains could be remarkably more powerful than digital neural networks if the have primitive functions that we haven't accounted for. For example, some networks seem to encode information in the firing rate , rather than just the presence of a signal. If neurons could, e.g. do frequency-based calculations (and not just threshold-based, like spiking neural nets), they could be orders of magnitude more…
the entropy rate goes way up if you consider spike timing dependent signals as well. but the difference in computational capacity between the brain and ML lies less in the brain's inherently time-dependent dynamics and more in the impressive computational capacity of single neurons. Dendrites compute, electrochemical dynamics during action potentials compute, synapses compute. All in complex time-dependent ways. check out izhikevich's dynamical systems in neuroscience for a taste of the computational capacity of the electrochemical dynamical system alone
Re: How deep is the brain? The shallow brain hypothesis
#175Earlier quoted context omitted.
AlexNet was the turning point for DL.
Why do you say that? Deep Learning was accelerating well before that (I would argue it has been accelerating for its entire existence). AlexNet was a state-of-the-art image recognition net for a (relatively) brief amount of time. It wasn't the first CNN to use GPU acceleration, and it was quickly eclipsed in terms of ImageNet performance. Regardless, I think bringing up AlexNet kinda invalidates your initial point. A…
is this schmidhuber's alt? sure they existed before AlexNet was where it really took off. just look at the number of citations. right paper, right time. CNNs were uniquely suited to the hardware at the time. because of their efficiency due to symmetry and suitability to GPGPU computing. not because of their history.
Re: How deep is the brain? The shallow brain hypothesis
#176Earlier quoted context omitted.
We can’t do that not because our mathematical neurons are too simple. We can’t do that because we don’t know the algorithms those biological neurons are running. Do you see the difference?
There is of course a difference between the two things you say. They're both the reason we can't recreate the brain in software though.
There are two separate goals: to simulate the brain in software, and to understand brain algorithms. They overlap, but they are still distinct, and appeal to different groups of people. Neuroscientists want to understand detailed brain operations. They are primarily interested in the brain itself. AI researchers want to understand intelligence, they are primarily interested in higher brain functions (e.g. reasoning, attention, short/long memory, emotions, motivations, goal setting, etc).
We can't (fully) recreate the brain in software partly because we don't know enough, and partly because it's too computationally complex - for example, we can't simulate an entire modern CPU at the transistor level - even though we know how each transistor works, and what each transistor does in the CPU - because each transistor requires a detailed physical model with hundreds of parameters. It's simply not computationally feasible using current supercomputers. Brain is even less feasible to simulate if we want to accurately simulate each individual neuron in it - even if we knew exactly how it works.
But the second goal is much more feasible, and we have made great progress simply by scaling up simple known algorithms which approximate some information processing functions in the brain (mainly pattern matching/prediction and attention). I can talk to GPT4 today just like I talk to other humans, and by the way, this is only possible because out of all AI/ML algorithms people have tried over the last 70 years, the most brain-like one have won (ANNs). If we want to make further progress in AI or if we want to make GPT5 to be more human-like (not sure we do), we don't necessarily need to simulate brain at a neuronal level, we simply need to understand a little bit more about higher level brain functions. Today, we (ML researchers) might actually benefit more from studying psychology than neuroscience.
Re: How deep is the brain? The shallow brain hypothesis
#177Beyond the mere topological metaphor of neural networks there is almost nothing in common between brains and widigital computation. This is a widespread fallacy of category.
> Beyond the mere topological metaphor of neural networks there is almost nothing in common between brains and widigital computation.
I mean, sure, but the topology is exactly what makes both work, so we only really care about the topology.
Re: How deep is the brain? The shallow brain hypothesis
#178Earlier quoted context omitted.
Being positive is not a requirement of commenting on HN, but you should comment with something that is substantive, so yes I do think you shouldn't have commented at all. Tone policing is cringe.
Exactly what are you doing here then? But hey I guess I can do this too. How's this? Using cringe as an adjective is cringe.
> But hey I guess I can do this too.
It sucks, doesn't it?
Re: How deep is the brain? The shallow brain hypothesis
#179Earlier quoted context omitted.
Have a read through the first paper describing a convolutional neural network, from 1998: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf There's absolutely no mention of biological inspiration whatsoever. At the same time, one can point to a long and rich history of convolutional filters being used in signal processing. And then there's the name, Convolutional Neural Network. The entire concept of a CNN is frame…
That is definitely not the first paper describing a CNN. That is not even the first paper by Le Cun describing CNNs (he was already on them as early as 1989[1]). Regardless, Le Cun is not the first to describe CNNs, merely one of the first to use them for OCR (specifically for hand-written text). The first neural network arch to use convolutions instead of matmuls was this[2], from the year of our lord 1988. This in…
I don't know why I'm still responding to this thread 24 hours later, but just thought I'd add this tweet from Le Cun: "Neuroscience greatly influenced me (there is a direct line from Hubel & Wiesel to ConvNets) and Geoff Hinton.
And the whole idea of neural nets and learning by adjusting synaptic weights clearly comes from neuroscience."
Re: How deep is the brain? The shallow brain hypothesis
#180Earlier quoted context omitted.
Have a read through the first paper describing a convolutional neural network, from 1998: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf There's absolutely no mention of biological inspiration whatsoever. At the same time, one can point to a long and rich history of convolutional filters being used in signal processing. And then there's the name, Convolutional Neural Network. The entire concept of a CNN is frame…
Surely you are trolling me now. There is a very clear biological inspiration mentioned in this paper: they literally define a CNN as having “receptive fields” and then they cite the same Hubel & Wiesel research mentioned before multiple times. LeCun mentions their research in papers even earlier in the 80s as well, during which they were awarded the Nobel prize for their research on the visual system. Of course there…