Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computational Power Found in the Arms of Neurons

quantamagazine.org

Post reply on HN
Print view View original

21–30 of 78 posts

Re: Computational Power Found in the Arms of Neurons

#21

Post by Elbrasch »

ASlave2Gravity wrote: post #14
valvar wrote: post #8

Earlier quoted context omitted.

You'll have to be more specific with what you mean by "machines". In what ways would machines be different from non-machines?

That's a good question. I guess I'd say a Turing machine is my definition of a machine. And I suppose the implication is that if we are machines, operating on a stream of input, does that make us deterministic? EDIT: What I'm ultimately interested in is if we, as humans, are operating in a realm that logic cannot operate in. If that makes any sense? I'll probably have to define realm!

As always, being deterministic may make a system predictable in theory, through in practice there are to many variables for it to matter. To soften the existential dread.

Re: Computational Power Found in the Arms of Neurons

#22

Post by tudorw »

prox wrote: post #9
tudorw wrote: post #2

“Brains may be far more complicated than we think,” as said by anyone who has worked with brains! Good stuff though, very interesting :)

As said by a brain studying brains

Don't get me started, if a complex brain can finally understand the brain, does that not mean we will still be missing the special sauce that was the cause of the complex comprehension, er, I'm out!

Re: Computational Power Found in the Arms of Neurons

#23

Post by visres »

buboard wrote: post #19
bonoboTP wrote: post #10

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

they are not. the current consensus in computational neuroscience is that a neuron can be equivalent to a 2-layer or 3-layer ANN with point nonlinearities. Edit: i am blocked from posting, so here are the refs: https://www.ncbi.nlm.nih.gov/pubmed/20800473 https://www.ncbi.nlm.nih.gov/pubmed/25554708 https://sci-hub.tw/https://www.sciencedirect.com/science/art...

Do you have any references you could share on this? I'd like to read more, thanks!

Re: Computational Power Found in the Arms of Neurons

#24

Post by nabla9 »

ASlave2Gravity wrote: post #14
valvar wrote: post #8

Earlier quoted context omitted.

You'll have to be more specific with what you mean by "machines". In what ways would machines be different from non-machines?

That's a good question. I guess I'd say a Turing machine is my definition of a machine. And I suppose the implication is that if we are machines, operating on a stream of input, does that make us deterministic? EDIT: What I'm ultimately interested in is if we, as humans, are operating in a realm that logic cannot operate in. If that makes any sense? I'll probably have to define realm!

Turing machine is mathematical model for computation. Deterministic Turing machine (DTM) is computationally equivalent to Non-Deterministic Turing machine (NTM) but the time complexity may not be the same.

>What I'm ultimately interested in is if we, as humans, are operating in a realm that logic cannot operate in. If that makes any sense?

Any mix of logic and randomness is still a machine. You must assume something spiritual if you think there exist rational action without computation behind it. Randomness does not help there.

Re: Computational Power Found in the Arms of Neurons

#25

Post by deepnet »

visres wrote: post #23
buboard wrote: post #19

Earlier quoted context omitted.

they are not. the current consensus in computational neuroscience is that a neuron can be equivalent to a 2-layer or 3-layer ANN with point nonlinearities. Edit: i am blocked from posting, so here are the refs: https://www.ncbi.nlm.nih.gov/pubmed/20800473 https://www.ncbi.nlm.nih.gov/pubmed/25554708 https://sci-hub.tw/https://www.sciencedirect.com/science/art...

Do you have any references you could share on this? I'd like to read more, thanks!

Minsky and Papert showed that computing XOR requires a 3 layer artificial neural net: Input, Hidden Layer, Output - which has 2 layers of weighted connections.

This[0] research shows a single neuron can compute XOR, thus the artificial neuron has less computational power than a real one.

[0] - https://www.reddit.com/r/MachineLearning/comments/ejbwvb/r_s...

Re: Computational Power Found in the Arms of Neurons

#26

Post by visres »

deepnet wrote: post #25
visres wrote: post #23

Earlier quoted context omitted.

Do you have any references you could share on this? I'd like to read more, thanks!

Minsky and Papert showed that computing XOR requires a 3 layer artificial neural net: Input, Hidden Layer, Output - which has 2 layers of weighted connections. This[0] research shows a single neuron can compute XOR, thus the artificial neuron has less computational power than a real one. [0] - https://www.reddit.com/r/MachineLearning/comments/ejbwvb/r_s...

Ah interesting thank you

Re: Computational Power Found in the Arms of Neurons

#27

Post by sansnomme »

svara wrote: post #11
bonoboTP wrote: post #10

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

This is an intuition that many neuroscientists have shared, it's just that direct experimental evidence of 1) it happening at all in a non-trivial way and 2) it being relevant to the behavior of an animal is somewhat scarce. Plus, the sum and threshold model actually gets you pretty far, as evidenced by artificial neural networks, so there might be some resistance to adding complexity that might not be necessary. Edi…

Also: spiking neural networks. Doing useful stuff with them has been a work-in-progress so far though.

Re: Computational Power Found in the Arms of Neurons

#28

Post by svara »

deepnet wrote: post #25
visres wrote: post #23

Earlier quoted context omitted.

Do you have any references you could share on this? I'd like to read more, thanks!

Minsky and Papert showed that computing XOR requires a 3 layer artificial neural net: Input, Hidden Layer, Output - which has 2 layers of weighted connections. This[0] research shows a single neuron can compute XOR, thus the artificial neuron has less computational power than a real one. [0] - https://www.reddit.com/r/MachineLearning/comments/ejbwvb/r_s...

That's exactly what TFA is about ;)

Re: Computational Power Found in the Arms of Neurons

#29

Post by chmod775 »

haffi112 wrote: post #5

I wonder if this is also the case for neurons in other animals. Probably it is, but it would be a stunning revelation if it is not the case.

From the top reddit comment:

> This paper is amazing. What is missing from the description above is that this is the first example of how human neurons are qualitatively different than rodent neurons (not only more computation power, but categorically different computation).

> ELI5: the way the biological human neuron implements XOR is by a formerly unknown type of local response to inputs, which is low below the threshold, maximal at the threshold and decreases as the input intensifies above the threshold. We never saw anything like that in any other animal. (link to the relevant figure from the paper)

Rodents being presumably the go-to non-human object of study.

Post reply on HN