Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep physical neural networks trained with backpropagation

nature.com

Post reply on HN
Print view View original

31–38 of 38 posts

Re: Deep physical neural networks trained with backpropagation

#31

Post by pmayrgundter »

whatshisface wrote: post #27
pmayrgundter wrote: post #26

Earlier quoted context omitted.

Ok, that's fair. That's i/o and yes, that's known to be highly linear by the time it gets to the efferent nerves, and makes sense it is before that as well. I think that still leaves the vast majority of the cortex using undefined mechanisms.

There's no need to hypothesize a wholly unique central nervous system signalling mechanism when, not only is the signalling mechanism of peripheral nerves understood, central nerves are observed doing the same thing.

I think it's fine as a hypothesis for the CNS, and my guess is it's correct for the spinal cord on up to the maybe the thalamus. But there the anatomy changes radically, as does the electrical activity.. eg the cluster waves (alpha, beta, theta) begin there and indicate some sort of group behaviors in various areas.

Afaik, we have correlative descriptions of what these waves indicate (importantly, they're associated with sleep, consciousness, attentiveness), but no direct mechanical model of them or a clear purpose. So yeah, spike timing could still be used at this level, but it seems other behaviors are also happening that may be more essential to the larger function.

Re: Deep physical neural networks trained with backpropagation

#32

Post by posterboy »

whatshisface wrote: post #27
pmayrgundter wrote: post #26

Earlier quoted context omitted.

Ok, that's fair. That's i/o and yes, that's known to be highly linear by the time it gets to the efferent nerves, and makes sense it is before that as well. I think that still leaves the vast majority of the cortex using undefined mechanisms.

There's no need to hypothesize a wholly unique central nervous system signalling mechanism when, not only is the signalling mechanism of peripheral nerves understood, central nerves are observed doing the same thing.

I do remember reading once that glia cells are not understood, and the something about how electro magnetic fields might also induce ... I do not remember it well because it wasn't very specific

The sentiment that synapses probably don't explain everything is rather common, anyhow. I'm thinking, the way the blood flow literally influences the relevant areas by transporting available energy for example, and neurotransmitters must be a very important point, and how those areas react in case of insufficiency would explain why I become nasty when tired and hungry at the same time.

Re: Deep physical neural networks trained with backpropagation

#33

Post by posterboy »

whatshisface wrote: post #16
dekhn wrote: post #13

Earlier quoted context omitted.

No. https://en.wikipedia.org/wiki/Epiphenomenalism

Note that that title ends in "ism," like "Calvinism," or "Evangelicalism."

hence epiphenomenal-ize? I don't thinkso.

Maybe it's more like epiphenomalis-m(a), where -is could be a genetive ending. I.e. the idea of epiphenomena.

Re: Deep physical neural networks trained with backpropagation

#35

Post by deepsun »

modeless wrote: post #25

Let me see if I can describe the laser part of the paper correctly. They made a laser pulse consisting of a bunch of different frequencies mixed together. The intensity of each frequency represents a controllable parameter of the system. The pulse was sent through a crystal that performs a complex transformation that mixes all the frequencies together in a nonlinear and noisy way. Then they measure the frequency spec…

> regular neural net

You used these words several times, and, considered title "physical neural networks", I always wondered if you mean regular like real, or like artificial. If it's artificial, I'm not sure which one of them is "regular" -- LSTM, full, transformers?

Re: Deep physical neural networks trained with backpropagation

#36

Post by modeless »

deepsun wrote: post #35
modeless wrote: post #25

Let me see if I can describe the laser part of the paper correctly. They made a laser pulse consisting of a bunch of different frequencies mixed together. The intensity of each frequency represents a controllable parameter of the system. The pulse was sent through a crystal that performs a complex transformation that mixes all the frequencies together in a nonlinear and noisy way. Then they measure the frequency spec…

> regular neural net You used these words several times, and, considered title "physical neural networks", I always wondered if you mean regular like real, or like artificial. If it's artificial, I'm not sure which one of them is "regular" -- LSTM, full, transformers?

I thought it was pretty clear in context that "regular neural net" was a short form of "conventional digital neural network" which I did spell out explicitly the first time.

Any type of artificial neural net could be used. LSTM, transformer, convolutional, fully connected, whatever you want.

Re: Deep physical neural networks trained with backpropagation

#37

Post by phreeza »

visarga wrote: post #29

If you can train a non-linear physical system with this method, in principle, you could also train real brains. You can't update the parameters of the brain, but you can inject signal. Assuming real brains to be black box functions for which you could learn a noisy estimator of gradients, it could be used for neural implants that supplement lost brain functionality, or a Matrix-like skill loading system.

You need a differentiable forward model of the process, which is not available for the human brain.

Re: Deep physical neural networks trained with backpropagation

#38

Post by amcoastal »

melissalobos wrote: post #5

> Deep-learning models have become pervasive tools in science and engineering. However, their energy requirements now increasingly limit their scalability.[1] They make this claim first, and cite one source. I haven't heard of this as an issue before. Is there anywhere else I could read more on this? [1] https://arxiv.org/abs/2104.10350

It's more of an inference and practical thing. If you want to equip something with limited energy (e.g. a drone using a small battery) with the ability to use a neural network for inference, their system could use much less energy than the typical computational setup.
Post reply on HN