Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Single cortical neurons as deep artificial neural networks

sciencedirect.com

Post reply on HN
Print view View original

31–40 of 135 posts

Re: Single cortical neurons as deep artificial neural networks

#31

Post by roenxi »

zetalyrae wrote: post #6
adrianN wrote: post #5

Earlier quoted context omitted.

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.

True, but transistors are designed by humans (for now), so you have an upper bound on complexity as well as a certain degree of modularity. Whereas biology has no such limitations because evolution doesn't care about the understandability of its designs. So I tend to err on the side of biology being more complex than not. But it's absolutely possible that the high-level behaviour is simple while the underlying implem…

Biology hasn't had time to figure out that intelligence is a good idea, up to humans the payoffs have been relatively low. The complexity in human intelligence probably focuses more on doing basic things extremely energy efficiently rather than being effective at thinking.

Humans can attempt to multiply numbers and frequently get the wrong result. That doesn't scream "pushing the limits of intelligence".

Re: Single cortical neurons as deep artificial neural networks

#32

Post by laurent92 »

freemint wrote: post #22

> Cortical neurons are well approximated by a deep neural network (DNN) with 5–8 layers I wonder how many cortical neurons it takes to approximate a ReLu or tanh well. I suspect this number being large than 1. If so the paper only shows an upper bound. Think how many neurons does it take to add a two 10 digit numbers. It is perfectly feasible that some (possibly large) part of this 5-8 layers is just "emulation overh…

That’s true, the brain uses a VM to run maths or science, only the best scientists succeed at understanding some of the concepts natively.

Same for music: A student runs the music sheet in a VM, and progressively JIT makes the movements native, which allows much faster execution, and which allows building on top of the base layer.

Maybe we’re doing it all wrong writing programs in assembler. We should give them to a VM, the VM should see the similarity between various pieces of the programs, make them inline, and we could teach the machine faster.

Re: Single cortical neurons as deep artificial neural networks

#33

Post by xpuente »

londons_explore wrote: post #3

This is trying to train a neural network to match the behaviour of various mathematical models of biological neurons. Said models still don't seem to be very close to being able to predict the behaviour of biological neurons though. If they did, then we'd have invented artificial human-level intelligence.

LN model not always works https://t.co/RNidCEXWcC?amp=1

Re: Single cortical neurons as deep artificial neural networks

#34

Post by nabla9 »

freemint wrote: post #22

> Cortical neurons are well approximated by a deep neural network (DNN) with 5–8 layers I wonder how many cortical neurons it takes to approximate a ReLu or tanh well. I suspect this number being large than 1. If so the paper only shows an upper bound. Think how many neurons does it take to add a two 10 digit numbers. It is perfectly feasible that some (possibly large) part of this 5-8 layers is just "emulation overh…

There is no need to approximate a ReLu or tanh well. Machine learning is statistical. The accuracy of these functions is not that important

ReLu is buggy and has an incorrect activation function for deep learning because it's not continuous everywhere. In practice, it rarely matters. It's chosen only because it's fast to implement buggy function than use someting proper.

The exact shape of tanh is not important either. It's enough to be monotone roughly s-shaped and easy to differentiate. Tanh is implemented in hardware so it's used.

Basically anything monotone and approximately differentiable works.

Re: Single cortical neurons as deep artificial neural networks

#35

Post by hoseja »

raducu wrote: post #20
hoseja wrote: post #9

Earlier quoted context omitted.

That doesn't follow at all. Human neurons aren't much different from mouse neurons, maybe chicken neurons, mosquito neurons. After you faithfully model a neuron you still need untold myriads of them and their interconnects to get human brain.

Also, sensations and feelings are not a logical/mathematical byproduct of the neurons; no matter how "well" you simulate "neurons", feelings and sensations will not emerge.

I don't think we have the faintest clue what subjective experience of self (whatever you call it, qualia?) actually is, to be able to say it isn't artificially reproducible.

Re: Single cortical neurons as deep artificial neural networks

#36

Post by vlovich123 »

adrianN wrote: post #29
vlovich123 wrote: post #24

Earlier quoted context omitted.

Human-level intelligence is highly variable. If we’re talking about something that can intelligently and independently make discoveries in math science and engineering, then the comparative advantage can’t stay long because you’d just pose the problem of “make yourself but faster” to it. It doesn’t even need discoveries. Imagine an AI that could manually transcode your high level code into absolutely optimal assembly…

> Imagine an AI that could manually transcode your high level code into absolutely optimal assembly The Halting Problem prevents such an AI from existing, but your point still stands of course.

How so?

Re: Single cortical neurons as deep artificial neural networks

#37

Post by mrfusion »

adrianN wrote: post #5
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.

Don’t forget that transistors are millions of times faster too.

Re: Single cortical neurons as deep artificial neural networks

#38

Post by twic »

nootropicat wrote: post #26
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

To add to that, there's evidence that may mean that memories are stored in DNA, shuffled between neurons in RNA capsids, evolutionary borrowed from retroviruses. That would make each neuron a complicated computer, functionally equivalent to one pc in capability and data storage. "The neuronal gene Arc is essential for long-lasting information storage in the mammalian brain, mediates various forms of synaptic plastici…

The passage you quote very much does not amount to "memories are stored in DNA", and as far as i know, memories are not stored in DNA sequence, and would be very surprised if they were.

What this mechanism does is take the transcription level of a gene in one cell, and induce a proportional signal in another cell. There are numerous mechanisms which do that in cells. Most of them are nowhere near as weird as Arc, but the net result is similar.

Re: Single cortical neurons as deep artificial neural networks

#39

Post by neuah »

nabla9 wrote: post #34
freemint wrote: post #22

> Cortical neurons are well approximated by a deep neural network (DNN) with 5–8 layers I wonder how many cortical neurons it takes to approximate a ReLu or tanh well. I suspect this number being large than 1. If so the paper only shows an upper bound. Think how many neurons does it take to add a two 10 digit numbers. It is perfectly feasible that some (possibly large) part of this 5-8 layers is just "emulation overh…

There is no need to approximate a ReLu or tanh well. Machine learning is statistical. The accuracy of these functions is not that important ReLu is buggy and has an incorrect activation function for deep learning because it's not continuous everywhere. In practice, it rarely matters. It's chosen only because it's fast to implement buggy function than use someting proper. The exact shape of tanh is not important eithe…

Also the nonlinearity only needs to be differentiable because ANNs are trained with gradient descent. With other more biologically plausible learning mechanisms, this might matter even less (or have other constraints / requirements)

Re: Single cortical neurons as deep artificial neural networks

#40

Post by adrianN »

vlovich123 wrote: post #36
adrianN wrote: post #29

Earlier quoted context omitted.

> Imagine an AI that could manually transcode your high level code into absolutely optimal assembly The Halting Problem prevents such an AI from existing, but your point still stands of course.

How so?

Suppose you write a program that simulates a TM and then prints "Halt" when the TM halts. The magic AI could optimize this program to just a print (or an infinite loop). This requires solving the halting problem.
Post reply on HN