Live data from Hacker News

Computational Power Found in the Arms of Neurons

quantamagazine.org

41–50 of 78 posts

Re: Computational Power Found in the Arms of Neurons

#41

For those of you that want to be horrified / amused by code, check out the materials and methods of the article itself. It contains a link to the full source code used to generate the modeling figures: https://senselab.med.yale.edu/ModelDB/showmodel.cshtml?model... The simulator used is called Neuron. Code for it is written in a custom language called Hoc and models are implemented in yet another domain specific lang…

Why horrified? A quick scan and it's actually nicely laid out and indented, split into logical sections and commented.

Without wanting to spend a lot of time on it, why is it horrific?

There's nothing intrinsically wrong with hardcoded values as they're biological constants, and nicely identified by the standard convention of using ALL_CAPS_VARIABLES.

Edit: Scanning a bit more, he's probably referring to this file, which is pretty horrific:

https://github.com/ModelDBRepository/254217/blob/master/_mor...

Re: Computational Power Found in the Arms of Neurons

#42

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

Using some back of the napkin math there are approximately a hundred trillion atoms in a typical neuron.

https://www.quora.com/If-a-neuron-were-a-galaxy-how-would-th...

Re: Computational Power Found in the Arms of Neurons

#43
post #5

I wonder if this is also the case for neurons in other animals. Probably it is, but it would be a stunning revelation if it is not the case.

From the top reddit comment: > This paper is amazing. What is missing from the description above is that this is the first example of how human neurons are qualitatively different than rodent neurons (not only more computation power, but categorically different computation). > ELI5: the way the biological human neuron implements XOR is by a formerly unknown type of local response to inputs, which is low below the thr…

Other differences of human neurons. I saw a presentation recently that said that populations of human neurons can synchronize to far higher frequencies than mouse neurons. Into the thousands kd hertz.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5100995/

Re: Computational Power Found in the Arms of Neurons

#44
post #8

Earlier quoted context omitted.

You'll have to be more specific with what you mean by "machines". In what ways would machines be different from non-machines?

That's a good question. I guess I'd say a Turing machine is my definition of a machine. And I suppose the implication is that if we are machines, operating on a stream of input, does that make us deterministic? EDIT: What I'm ultimately interested in is if we, as humans, are operating in a realm that logic cannot operate in. If that makes any sense? I'll probably have to define realm!

You could argue that "man-made machines", "biological machines" (all of life, animal, vegetal and fungus) and "simple matter machines" (like stars are engines or telluric planets are combustion heaters), even the "surface natural ecosystem of Earth", are all different material implementations of machines: organized systems. Carbon-based, copper-based, hydrogen-based... you might map the whole periodic table of elements minus the rare column perhaps.

Ultimately you'd find a common unicity — like electrical charges, strong/weak force, arrangements of "gates" and "structure" etc. — but used in different ways (for instance iirc charged clouds of gas in space don't form new stars because gravity at close range is weaker than the charge that repels them, only strong enough at larger distances to hold them together; that's dramatically different from the use of charges in biological cells).

The cosmos itself is but a big machine. Who's to say we (I mean the whole planet, maybe star system, maybe galaxy itself) aren't actually just a single "cell" of the cosmos? That we are part of a much bigger machine, that we are like those processing units on dendrites in the article, if the universe is a big brain of sorts?[1]

These are all types of machines (X. process of information; Y. engines to convert energy; Z. structures that "restrict" "flow" like gates, transistors, cell membranes, dendrites; N...), which apparently may be expressed, materially implemented in different ways.

To take your question about logic.

If you mean the universal logic exposed above, I don't think so personally — merely because there's no evidence for it whatsoever, no observation; and there's also no need if we accept that from such "machinery" complexity may emerge complex systems (e.g. humans).

If you mean logic from within the human mind — and this begs the question of whether maths and physics are "invented" or "discovered" in the background — then we must assume a subjective answer, at best an aggregate of "non-disproved facts" that we can all agree on within normalcy (edge cases helping us understand said average norm).

"Logic" is but one of several operating modes. Are we more than that? Most certainly yes. That's the experience of all of us. Because perception is so centered on thought, we tend to overappreciate the importance of thinking in our lives, in our behaviors and even values; but the relative picture gradually revealed by biology and psychology and sociology and economics etc. is that we are mostly irrational, mostly automatic (trained habits, patterns recognition, educated intuition, etc), and actually very little in the way of "logical behaviors units".

I'll let you ponder the discrepancy from an absolutely (perhaps) low-level logical machine to the possibility of a chaotic non-logical emergent high-level behavior. What it means for life, for AI, and possibly much bigger or smaller things we consider "inert", for now.

[1]: I mean, look at the larger structures of the universe, tell me it doesn't look like a sample from a biological tissue... http://cosmology.com/GalacticWalls.html (figure 6: http://cosmology.com/images/darkmatterdistribution.jpg)

Re: Computational Power Found in the Arms of Neurons

#45
post #40

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

The idea is that a certain simplification could work enough to be valuable. It is similar with gravity isn't it? We don't have a grand unified theory of quantum mechanics, but even Newton's abstractions serves us well in certain situations. We don't know how it behaves in the micro level but we can still observe the macro effects and create an abstraction out of said behavior. And to be honest, it took us somewhere .…

The risk is that we've developed a formalism and ecosystem that works on entirely different principles than the brain, even if it looks similar.

It still works, but it might not be a useful model to anyone studying the brain. I've yet to meet the neuroscientist who assumes it is, so perhaps that's not a problem.

Re: Computational Power Found in the Arms of Neurons

#46

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

Around the time everyone was talking about tensegrity there was a researcher that had figured out a mechanism by which cells can sense pressure, due I think to mechanical stress on the molecules in membrane being detectable inside the cell. Proteins and lipids as proverbial nerve endings. They speculated from this that neurons rely on quantum effects, possibly using similar mechanisms, and that the surface area is involved.

The arms of neurons have a lot of the surface area, don’t they?

Re: Computational Power Found in the Arms of Neurons

#47
Ah, it's spiking edges as opposed to spiking node activations. I like this because it applies to 1 of many connections uniquely, rather than dividing the entire graph by spiking neurons. Exponentially more edges than nodes in a dense net. Connections are always more important than entities.

```The dendrites generated local spikes, had their own nonlinear input-output curves and had their own activation thresholds, distinct from those of the neuron as a whole```

How would this work in practice? Apply activations to multiplication values of the weight, or just don't perform the multiplication if the activation of the node is low?

Re: Computational Power Found in the Arms of Neurons

#48

Question for ML people: would backpropagation work if some of the neurons had nonmonotonic activation functions (for example exp(-x²)), or would the gradient descent get stuck on one side or the other?

It would probably work with careful choice of learning rate, initialization, and weight decay to keep signals small. Batch norm would play a larger role (probably want to use it after the activation fn). I don't see why it would get stuck on either side, but it could obviously get stuck if enough signals grow too large on both sides.

Re: Computational Power Found in the Arms of Neurons

#49
post #5

I wonder if this is also the case for neurons in other animals. Probably it is, but it would be a stunning revelation if it is not the case.

From the top reddit comment: > This paper is amazing. What is missing from the description above is that this is the first example of how human neurons are qualitatively different than rodent neurons (not only more computation power, but categorically different computation). > ELI5: the way the biological human neuron implements XOR is by a formerly unknown type of local response to inputs, which is low below the thr…

Could you link the Reddit comment please?

Re: Computational Power Found in the Arms of Neurons

#50

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

Understanding of neurons goes wayy beyond that today, see https://en.wikipedia.org/wiki/Hodgkin%E2%80%93Huxley_model

ANNs are abstractions that seem to work. We still don't have a full understanding of how two neurons communicate.

Post reply on HN