Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computational Power Found in the Arms of Neurons

quantamagazine.org

Post reply on HN
Print view View original

31–40 of 78 posts

Re: Computational Power Found in the Arms of Neurons

#31

Post by deckar01 »

> It may also prompt some computer scientists to reappraise strategies for artificial neural networks, which have traditionally been built based on a view of neurons as simple, unintelligent switches.

It has been suggested that neural networks have diverged from biological principles to the point where biological research does not provide any useful improvement to current machine learning techniques. Back propagation of errors was a major advancement in the 1970s, and it was developed purely based on mathematical principles like statistics, differential equations, and calculus. As far as I have read, error back propagation is not how biological NNs learn, and it seems to be a much more efficient strategy. Biology seems much more brute force in comparison.

Re: Computational Power Found in the Arms of Neurons

#32

Post by ASlave2Gravity »

Elbrasch wrote: post #21
ASlave2Gravity wrote: post #14

Earlier quoted context omitted.

That's a good question. I guess I'd say a Turing machine is my definition of a machine. And I suppose the implication is that if we are machines, operating on a stream of input, does that make us deterministic? EDIT: What I'm ultimately interested in is if we, as humans, are operating in a realm that logic cannot operate in. If that makes any sense? I'll probably have to define realm!

As always, being deterministic may make a system predictable in theory, through in practice there are to many variables for it to matter. To soften the existential dread.

Existential dread is most definitely softened! Thank you.

Re: Computational Power Found in the Arms of Neurons

#33

Post by ASlave2Gravity »

nabla9 wrote: post #24
ASlave2Gravity wrote: post #14

Earlier quoted context omitted.

That's a good question. I guess I'd say a Turing machine is my definition of a machine. And I suppose the implication is that if we are machines, operating on a stream of input, does that make us deterministic? EDIT: What I'm ultimately interested in is if we, as humans, are operating in a realm that logic cannot operate in. If that makes any sense? I'll probably have to define realm!

Turing machine is mathematical model for computation. Deterministic Turing machine (DTM) is computationally equivalent to Non-Deterministic Turing machine (NTM) but the time complexity may not be the same. >What I'm ultimately interested in is if we, as humans, are operating in a realm that logic cannot operate in. If that makes any sense? Any mix of logic and randomness is still a machine. You must assume something…

> You must assume something spiritual if you think there exist rational action without computation behind it.

I'm still uncertain i.e. spiritual. I'm finding, as I grow older, logic and reason fail to answer for my experiences. Whether that's a shortcoming in my own ability to comprehend logic and reason, or because logic and reason simply do not have all the answers, I do not know.

> Randomness does not help there.

The human condition, one might argue, emerged as a way to handle randomness. I suppose, at some level, if we were to remove all randomness from the universe, life would be pretty pointless (insomuch as life is not already pointless.)

Re: Computational Power Found in the Arms of Neurons

#34

Post by telmo »

deckar01 wrote: post #31

> It may also prompt some computer scientists to reappraise strategies for artificial neural networks, which have traditionally been built based on a view of neurons as simple, unintelligent switches. It has been suggested that neural networks have diverged from biological principles to the point where biological research does not provide any useful improvement to current machine learning techniques. Back propagation…

> Biology seems much more brute force in comparison.

Biology is more decentralized for sure, but I wouldn't be so fast with calling it "brute force". One interesting feature of biological neural networks is the variety of neurotransmitters. In very simple terms, they are not only capable of exciting or inhibiting neurons from firing, but they are also capable of regulating how neurons adapt to the stimulae they have been exposed to. In other words, the biological network not only learns, but it runs the learning algorithm itself. It appears to do this by creating various layers of communication with different effects on the computing nodes.

In my view, we have not been smart enough yet to figure out nature's algorithm.

Re: Computational Power Found in the Arms of Neurons

#36

Post by nravic »

sansnomme wrote: post #27
svara wrote: post #11

Earlier quoted context omitted.

This is an intuition that many neuroscientists have shared, it's just that direct experimental evidence of 1) it happening at all in a non-trivial way and 2) it being relevant to the behavior of an animal is somewhat scarce. Plus, the sum and threshold model actually gets you pretty far, as evidenced by artificial neural networks, so there might be some resistance to adding complexity that might not be necessary. Edi…

Also: spiking neural networks. Doing useful stuff with them has been a work-in-progress so far though.

Yeah, spiking neural networks are a tough nut to crack. Check out Nengo if you're interested in learning more

Re: Computational Power Found in the Arms of Neurons

#37

Post by orbifold »

For those of you that want to be horrified / amused by code, check out the materials and methods of the article itself. It contains a link to the full source code used to generate the modeling figures:

https://senselab.med.yale.edu/ModelDB/showmodel.cshtml?model...

The simulator used is called Neuron. Code for it is written in a custom language called Hoc and models are implemented in yet another domain specific language. Hardcoded parameters galore including whole sections of code that are prefaced with: this has been copied from this other publication.

https://senselab.med.yale.edu/ModelDB/showmodel.cshtml?model...

The dendritic tree geometry is specified as hardcoded set of point and branch declarations

https://senselab.med.yale.edu/ModelDB/showmodel.cshtml?model...

(apparently that morphology was reconstructed from imaging data?)

Re: Computational Power Found in the Arms of Neurons

#38

Post by nabla9 »

bonoboTP wrote: post #10

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

Inside each sell is gene regulatory network (GRN). Active genes produce proteins. Some of those proteins increase or decrease activity of genes. GRN as a whole can be modeled as stochastic recurrent neural network.

The potential for computation inside each neuron is similar to recurrent neural network.

Re: Computational Power Found in the Arms of Neurons

#39

Post by prox »

tudorw wrote: post #22
prox wrote: post #9

Earlier quoted context omitted.

As said by a brain studying brains

Don't get me started, if a complex brain can finally understand the brain, does that not mean we will still be missing the special sauce that was the cause of the complex comprehension, er, I'm out!

This is why in some philosophies, there is a big difference between experiential knowledge and awareness and empirical ‘measurement’ knowledge.

Re: Computational Power Found in the Arms of Neurons

#40

Post by dkarras »

bonoboTP wrote: post #10

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

The idea is that a certain simplification could work enough to be valuable. It is similar with gravity isn't it? We don't have a grand unified theory of quantum mechanics, but even Newton's abstractions serves us well in certain situations. We don't know how it behaves in the micro level but we can still observe the macro effects and create an abstraction out of said behavior.

And to be honest, it took us somewhere. Yes we don't have AGI or anything close to it but the products of machine learning is something we depend on every single day now.

Post reply on HN