Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Single cortical neurons as deep artificial neural networks

sciencedirect.com

Post reply on HN
Print view View original

91–100 of 135 posts

Re: Single cortical neurons as deep artificial neural networks

#91

Post by gragundier »

sdenton4 wrote: post #65
causi wrote: post #63

Earlier quoted context omitted.

Indeed. Building an AI that matches human intelligence using equal or less mass than a human brain requires one or both of two things to be true: 1. The computational mass efficiency of brain tissue is very far from optimum. Considering the amount of time evolution has been improving upon it, I highly doubt that is true. 2. Most of the brain's computation is not involved in cognition. That may be true. We don't reall…

/2. Most of the brain's computation is not involved in cognition. That may be true. We don't really know./ I thought this at least was fairly well understood: We do in fact use our whole brains, as anything less would be a fantastic waste of resources, which evolution would have taken care of long ago. We have numerous human-specific adaptations to deal with the relatively massive brains we're carrying around.

Not necessarily. If a brain provides enough of a evolutionary advantage, it might still be a wondrously unoptimal hacky piece of sphagetti.

Re: Single cortical neurons as deep artificial neural networks

#92

Post by datameta »

api wrote: post #78
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Older estimates are probably short by multiple orders of magnitude. We can see this in practice by looking at e.g. a self-driving Tesla vs. a mouse. Watch a frightened mouse run across an uneven landscape, climbing and clambering and avoiding obstacles as it controls four independent limbs and countless small muscles in real time. A Tesla's auto-drive is nowhere near that good in spite of having only a few levers to…

Perhaps Spot from Boston Dynamic is a more fair SotA comparison. Though even then, the magnitude of scale is off and still the gulf in ability is wide. I wonder how our artificial insects stack up in comparison to organic ones computationally. I think there is much to learn from the bee brain, for instance.

Relevant read: "The thermodynamic efficiency of computations made in cells across the range of life" https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5686401/

Re: Single cortical neurons as deep artificial neural networks

#94

Post by vlovich123 »

simiones wrote: post #55
hyper_dynamics wrote: post #52

Earlier quoted context omitted.

It might not be able to do it for any turing machine/ a universal turing machine – but it might quickly figure out what a turing machine will do without executing all steps of it.

It might get it right some of the time, but it will be necessarily wrong some of the time. It's also very possible that for many (possibly even most) TMs, the most efficient algorithm for predicting the output is that TM itself.

I think the problem is you're assuming that general AI = Turing machine, but there's no indication that needs to be the case.

"General AI" to me means human intelligence running on an artificial system (silicon, simulated brain, etc), so the optimization I'm thinking of is more akin to having an assembly expert translate your code into assembly than a compiler optimization pass.

Given that I have optimized my fair bit of code by removing abstraction layers or simplifying code, by definition a general AI should be similarly capable & can handle even ambiguous tasks like "refactor this codebase in this way". Obviously this gives up accuracy, but humans make mistakes writing code as well & it would be much easier to say "I've observed a fault that has this properties. Figure out the problem". It should do an even better job than I can on problems like that because for complicated problems it should be able to follow complex codebases with greater ease than I.

Again, I'm defining a tautological definition of "general AI" as one that's capable of doing all that. If it's not capable of doing that then it's not general AI.

Re: Single cortical neurons as deep artificial neural networks

#95

Post by joe_the_user »

Cortical neurons are well approximated by a deep neural network (DNN) with 5–8 layers

This is a link to the abstract so you don't get to find out the width of these networks. But given that neural networks are universal approximators, it seems to me that all that's being said is a neuron is a very complicated thing. And so the phrasing seems to give an unjustified impression you still best think of a biological neural network as just a larger artificial neural network.

Re: Single cortical neurons as deep artificial neural networks

#96

Post by orbifold »

nabla9 wrote: post #15
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Your terminology is inaccurate. Threshold neurons like Pitts-McCulloch neurons are rarely used in artificial neural network architectures. They can't learn. ANN's uses a nonlinear activation function that is differentiable (at least in practice). That small change makes huge difference.

That is not true, one can easily regularise the derivative, which is delta distribution, in some appropriate way, the easiest one being a 'triangle' centered at zero. That way one can actually easily train networks of McCulloch-Pitts neurons.

Re: Single cortical neurons as deep artificial neural networks

#97

Post by orbifold »

sdenton4 wrote: post #66
shmageggy wrote: post #54

Earlier quoted context omitted.

No, because it's wrong. Threshold neurons are still differentiable almost everywhere, no different than Relus which are ubiquitous. They may not be very good activation functions but they don't prevent a network from learning.

Sorry, but I think I disagree... If I understand the PM neuron, it's outputs are boolean. This means there's no backprop signal, despite differentiability, since the outputs are constant (thus, gradient zero) in any neighborhood, which in turns zeros out any learning signal you would want to backprop through them. So you need a different learning strategy than backprop to use them. (see also: the 'dead neuron' proble…

One can regularise the derivative (a delta distribution) in several ways (e.g. a triangle at zero) and that is good enough (even from a theoretical perspective) to find an approximate gradient. Experimentally it is then possible to train deep neural networks with such non-linearities.

Re: Single cortical neurons as deep artificial neural networks

#99

Post by vikramkr »

caddemon wrote: post #73
vikramkr wrote: post #64

Earlier quoted context omitted.

In dna definitely implies in dna sequence. Especially with dna storage becoming a thing, and with aspects of our physiology actually using dna modifications for their function and to "remember" things (e.g. VDJ recombination), its best not to confuse epigenetic vs genetic information storage. Each hypothesis implies different things and different followup questions.

They are certainly different things, but that's why I think more specific language should be used when the difference matters. The other comment was trying to make a more abstract point about what sort of memories an individual neuron can store using DNA, the main point wouldn't really be changed if that was through purely epigenetic mechanisms. I don't think the actual argument he made was compelling, but the claim…

Perhaps, but I'd argue the differences do matter quite a bit to the computational implications, especially as it relates to ability to reprogram, state space, and persistence, and I think there would be a much different reaction if I made a post mixing up data on a computer's hard drive with the state of whether each transistor in a cpu is on or off, or something like that.

Re: Single cortical neurons as deep artificial neural networks

#100

Post by abeppu »

causi wrote: post #63
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Indeed. Building an AI that matches human intelligence using equal or less mass than a human brain requires one or both of two things to be true: 1. The computational mass efficiency of brain tissue is very far from optimum. Considering the amount of time evolution has been improving upon it, I highly doubt that is true. 2. Most of the brain's computation is not involved in cognition. That may be true. We don't reall…

I think the idea that brain tissue is near optimally efficient is interesting. Yes, it's had a long time to evolve. But the same can be said about photosynthesis which is less efficient at capturing solar energy than PVs. The evolution of brain tissue was under constraints about something that could be made by biological systems from the resources we could eat. Is it not plausible that some very efficient computational substrate can be made, but requires minerals and chemical and industrial processes which would be toxic or impossible for life?
Post reply on HN