Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Single cortical neurons as deep artificial neural networks

sciencedirect.com

Post reply on HN
Print view View original

11–20 of 135 posts

Re: Single cortical neurons as deep artificial neural networks

#11

Post by zetalyrae »

singularity2001 wrote: post #8
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Not to speak of all the 'weights' which are stored internaly in DNA methylation, among other memory forming mechanisms (proven recently). (part of the quadrillion atoms system of course)

And if you need ab-initio quantum chemistry (or equivalent) simulations of every atom in a human brain, the timelines for both AGI and whole brain emulation get pushed far, far into the future.

Assuming AGI requires about as much hardware as it'd take to emulate a brain, at least.

Re: Single cortical neurons as deep artificial neural networks

#12

Post by cblconfederate »

"The spiking times of L5 cortical neurons without NMDA receptors can sufficiently be approximated with a 5-8 layer DNN"

This isn't as novel as it sounds, previous work has modeled e.g. firing rates with 2 or 3 layer ANNs, nor does it provide some fundamental insight imho

Re: Single cortical neurons as deep artificial neural networks

#13

Post by yarcob »

adrianN wrote: post #5
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.

But digital chips are designed to use transistors as on/off switches, so the complicated physics don't matter for understanding a microprocessor. (It's different for analog ICs, where the characteristics of the individual transistors do matter)

Re: Single cortical neurons as deep artificial neural networks

#14

Post by jgilias »

londons_explore wrote: post #3

This is trying to train a neural network to match the behaviour of various mathematical models of biological neurons. Said models still don't seem to be very close to being able to predict the behaviour of biological neurons though. If they did, then we'd have invented artificial human-level intelligence.

Arguably, you'd need to simulate tens of billions of neurons in order to achieve human-level intelligence. So, even if you'd have correctly simulated a single neuron, there's still a lot to cover to achieve human-level AI. And even then. The few unfortunate cases where a human child has been reared outside of the normal environment (see Genie), having a human brain has turned out not to be enough to have intelligence that would be recognized as 'human-level'. So, apart from the simulated brain itself, you'd need to devise an appropriate training environment to use for training said brain to achieve human-level intelligence. Which would be a formidable feat in and of itself.

Re: Single cortical neurons as deep artificial neural networks

#15

Post by nabla9 »

zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Your terminology is inaccurate.

Threshold neurons like Pitts-McCulloch neurons are rarely used in artificial neural network architectures. They can't learn.

ANN's uses a nonlinear activation function that is differentiable (at least in practice). That small change makes huge difference.

Re: Single cortical neurons as deep artificial neural networks

#16

Post by danwills »

in3d wrote: post #7
Qu3tzal wrote: post #4

5-8 layers but how many weights per layer? I can’t access the article but it says it’s a deep CNN so probably in the order of a few thousands weights per layer.

Up to 256.

Is that 256 weights per-cell per-layer, or 256 in total? Is "how many cells in each layer" a valid question too?

Re: Single cortical neurons as deep artificial neural networks

#17

Post by zetalyrae »

nabla9 wrote: post #15
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Your terminology is inaccurate. Threshold neurons like Pitts-McCulloch neurons are rarely used in artificial neural network architectures. They can't learn. ANN's uses a nonlinear activation function that is differentiable (at least in practice). That small change makes huge difference.

I stand corrected. But the point is that artificial neurons are computationally simple.

Re: Single cortical neurons as deep artificial neural networks

#18

Post by raducu »

jgilias wrote: post #14
londons_explore wrote: post #3

This is trying to train a neural network to match the behaviour of various mathematical models of biological neurons. Said models still don't seem to be very close to being able to predict the behaviour of biological neurons though. If they did, then we'd have invented artificial human-level intelligence.

Arguably, you'd need to simulate tens of billions of neurons in order to achieve human-level intelligence. So, even if you'd have correctly simulated a single neuron, there's still a lot to cover to achieve human-level AI. And even then. The few unfortunate cases where a human child has been reared outside of the normal environment (see Genie), having a human brain has turned out not to be enough to have intelligence…

Sensations cannot be produced, no matter how many neurons you simulate.

We can only model certain phenomena we deem important, but who can really say?

I'm happy with "less than human" simulations, in fact, I'd be disgusted with a perfect human simulation -- if they help us automate/solve problems.

Re: Single cortical neurons as deep artificial neural networks

#19

Post by throwaway316943 »

adrianN wrote: post #5
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.

I think even dramatically simpler might be an understatement. Neurons are cells, living organisms capable of growth, movement, and some fairly intelligent interaction with their environment. A transistor doesn’t even approach that, it’s more on the level of a single protein within a neuron and not a very complex one.

Re: Single cortical neurons as deep artificial neural networks

#20

Post by raducu »

hoseja wrote: post #9
londons_explore wrote: post #3

This is trying to train a neural network to match the behaviour of various mathematical models of biological neurons. Said models still don't seem to be very close to being able to predict the behaviour of biological neurons though. If they did, then we'd have invented artificial human-level intelligence.

That doesn't follow at all. Human neurons aren't much different from mouse neurons, maybe chicken neurons, mosquito neurons. After you faithfully model a neuron you still need untold myriads of them and their interconnects to get human brain.

Also, sensations and feelings are not a logical/mathematical byproduct of the neurons; no matter how "well" you simulate "neurons", feelings and sensations will not emerge.
Post reply on HN