Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computational Power Found in the Arms of Neurons

quantamagazine.org

Post reply on HN
Print view View original

61–70 of 78 posts

Re: Computational Power Found in the Arms of Neurons

#61

Post by buboard »

nabla9 wrote: post #38
bonoboTP wrote: post #10

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

Inside each sell is gene regulatory network (GRN). Active genes produce proteins. Some of those proteins increase or decrease activity of genes. GRN as a whole can be modeled as stochastic recurrent neural network. The potential for computation inside each neuron is similar to recurrent neural network.

Except the clock is ticking every 30 minutes in that network. it's too slow for behavior. GRNs are relevant to long-term plasticity which is protein-dependent and has prominent modulators (e.g. CREB)

Re: Computational Power Found in the Arms of Neurons

#62

Post by [deleted] »

dkarras wrote: post #40
bonoboTP wrote: post #10

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

The idea is that a certain simplification could work enough to be valuable. It is similar with gravity isn't it? We don't have a grand unified theory of quantum mechanics, but even Newton's abstractions serves us well in certain situations. We don't know how it behaves in the micro level but we can still observe the macro effects and create an abstraction out of said behavior. And to be honest, it took us somewhere .…

[deleted]

Re: Computational Power Found in the Arms of Neurons

#63

Post by buboard »

orbifold wrote: post #37

For those of you that want to be horrified / amused by code, check out the materials and methods of the article itself. It contains a link to the full source code used to generate the modeling figures: https://senselab.med.yale.edu/ModelDB/showmodel.cshtml?model... The simulator used is called Neuron. Code for it is written in a custom language called Hoc and models are implemented in yet another domain specific lang…

The morphology files are automatically generated by a 3d reconstruction program called neurolucida. NEURON is an ancient program, written in 1983, using a custom parser for the ad-hoc language literally called hoc. It has a hardcoded limit in the number of lines a function can have (hence you have to split func1(), func2() etc) and a gazillion bugs. It uses a modified kinetic equation solver which compiles those .mod files to to a DLL that NEURON then uses to run. It uses some GUI library called InterViews from god knows where. It's generally a rather bad program nowadays, but written by the guy who invented a faster Crank-nicholson method to integrate the cable equations (Hines). Despite a few alternatives existing, NEURON is still the most popular because there are a lot of ion mechanism kinetic modules (the mod files) that you can download and reuse from previously published models, which have been hand-tuned to be quasi-realistic. NEURON is supposed to be used by neurophysiologists, not by programmers.

I think a mass open source campaign to replace NEURON and ModelDB with something more modern, and clean up and convert every published model would be worth the effort. There are quite a few alternative programs (GENESIS, Neuroconstruct etc) that are either incomplete or abandoned because of lack of funding/interest.

Re: Computational Power Found in the Arms of Neurons

#65

Post by bnegreve »

Filligree wrote: post #45
dkarras wrote: post #40

Earlier quoted context omitted.

The idea is that a certain simplification could work enough to be valuable. It is similar with gravity isn't it? We don't have a grand unified theory of quantum mechanics, but even Newton's abstractions serves us well in certain situations. We don't know how it behaves in the micro level but we can still observe the macro effects and create an abstraction out of said behavior. And to be honest, it took us somewhere .…

The risk is that we've developed a formalism and ecosystem that works on entirely different principles than the brain, even if it looks similar. It still works, but it might not be a useful model to anyone studying the brain . I've yet to meet the neuroscientist who assumes it is, so perhaps that's not a problem.

> The risk is that we've developed a formalism and ecosystem that works on entirely different principles [...]

It's a possibility, but I sometime have the feeling that people dismiss the idea that such a simple model of the brain could be enough to explain complex behaviors because they want to believe there is more to the brain.

I'm sure real neurons are very complex and difficult to model, but I also believe that the real challenge is to explain how neurons interact with each other, not how they behave individually.

Re: Computational Power Found in the Arms of Neurons

#66

Post by ASlave2Gravity »

aratakareigen wrote: post #55
ASlave2Gravity wrote: post #33

Earlier quoted context omitted.

> You must assume something spiritual if you think there exist rational action without computation behind it. I'm still uncertain i.e. spiritual. I'm finding, as I grow older, logic and reason fail to answer for my experiences. Whether that's a shortcoming in my own ability to comprehend logic and reason, or because logic and reason simply do not have all the answers, I do not know. > Randomness does not help there.…

What is the purpose of life, which you say depends on randomness?

Well, the future as a mystery, something to be discovered, not known in advance, depends on randomness. I would argue if life weren't random and we could see past, present, and future with perfect clarity, it would be a little pointless. Every action you took, you'd know the outcome before taking the action. It's boring.

There is no purpose of life, in my opinion. But with randomness it's a lot more intresting.

Re: Computational Power Found in the Arms of Neurons

#67

Post by ASlave2Gravity »

K0SM0S wrote: post #44
ASlave2Gravity wrote: post #14

Earlier quoted context omitted.

That's a good question. I guess I'd say a Turing machine is my definition of a machine. And I suppose the implication is that if we are machines, operating on a stream of input, does that make us deterministic? EDIT: What I'm ultimately interested in is if we, as humans, are operating in a realm that logic cannot operate in. If that makes any sense? I'll probably have to define realm!

You could argue that "man-made machines", "biological machines" (all of life, animal, vegetal and fungus) and "simple matter machines" (like stars are engines or telluric planets are combustion heaters), even the "surface natural ecosystem of Earth", are all different material implementations of machines: organized systems. Carbon-based, copper-based, hydrogen-based... you might map the whole periodic table of elemen…

> Because perception is so centered on thought, we tend to overappreciate the importance of thinking in our lives, in our behaviors and even values

Very true. Thought can become our master easily. As a programmer, I often find myself struggling to get out of 'logic' mode or 'thinking' mode.

> I'll let you ponder the discrepancy from an absolutely (perhaps) low-level logical machine to the possibility of a chaotic non-logical emergent high-level behavior. What it means for life, for AI, and possibly much bigger or smaller things we consider "inert", for now.

Wonderfully put! I think the ideas of emergent behaviours are a saving grace for the incessant desire of ours to reduce everything to its atomic (in the sense of an atomic operation, not an atom) components. The emergent behaviours, can't be predicted. They can't be algorithmized. Becuase to build algorithms one must go down to those atomic operations.

My wider concern is that we are currently obsessed with analytical thought and philosophy and that the continental philosophy has become a 2nd class citizen. I think this has happened becuase analytical thought lends itself to being algorithmized, whereas continental philosophy does not.

Sorry, that was a bit of a tangent!

Yes! Love the pictures! I have had the same thought/feeling when I first saw them too!

Re: Computational Power Found in the Arms of Neurons

#68

Post by salty_biscuits »

bonoboTP wrote: post #10

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

Computing weighted sums and thresholds gives you a universal function approximator, so that's nice. I'm more interested in how the buggers set those weights and thresholds. It is hard to imagine them having access to any gradient information, so how do they work?

Re: Computational Power Found in the Arms of Neurons

#69

Post by AstralStorm »

salty_biscuits wrote: post #68
bonoboTP wrote: post #10

Knowing little about neuroscience outside of biology class, I've always wondered why neurons would be so simple as to just compute weighted sums and thresholds. I mean there are unicellular creatures that can do incredibly complex things, and neurons, as all cells, are ultimately the descendants of such unicellular organisms, "teamed up" into a multi-cellular creature. They have a lot of internal structure, organelle…

Computing weighted sums and thresholds gives you a universal function approximator, so that's nice. I'm more interested in how the buggers set those weights and thresholds. It is hard to imagine them having access to any gradient information, so how do they work?

They do have chemical gradients, but the mechanisms is not even comparable to simple backpropagation...

Re: Computational Power Found in the Arms of Neurons

#70

Post by AstralStorm »

bnegreve wrote: post #65
Filligree wrote: post #45

Earlier quoted context omitted.

The risk is that we've developed a formalism and ecosystem that works on entirely different principles than the brain, even if it looks similar. It still works, but it might not be a useful model to anyone studying the brain . I've yet to meet the neuroscientist who assumes it is, so perhaps that's not a problem.

> The risk is that we've developed a formalism and ecosystem that works on entirely different principles [...] It's a possibility, but I sometime have the feeling that people dismiss the idea that such a simple model of the brain could be enough to explain complex behaviors because they want to believe there is more to the brain. I'm sure real neurons are very complex and difficult to model, but I also believe that t…

And why do you believe the current non-intelligent ML is sufficient?

As it is, ML is running away rather fast from the integrator model by introducing explicit gating and nonlinearities in the neurons.

Yet it is still a toy compared to a nematode.

Post reply on HN