Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teaching physics to neural networks removes 'chaos blindness'

phys.org

Post reply on HN
Print view View original

61–70 of 81 posts

Re: Teaching physics to neural networks removes 'chaos blindness'

#61

Post by cjfd »

mtgp1000 wrote: post #25
Fricken wrote: post #15

Earlier quoted context omitted.

Don't sell biology short like that. Human driver are born with a mental model of physics that's been honed 24 hours a day since before they were diatoms.

I don't think that's quite right. I believe that humans are essentially born as blank neural networks; it's the structure, and the graph of connections between brain structures and sensory inputs, that is effectively primed for learning certain tasks that we find to be intuitive. A baby is not born with the knowledge of body movement, for example, but through natural exploration of the body and environment, almost al…

This is clearly wrong. If you put your pinky finger inside the hand of a baby s/he will grab it and hold it. This can be seen very early.

Re: Teaching physics to neural networks removes 'chaos blindness'

#62

Post by DrAwdeOccarim »

mywittyname wrote: post #9

Why do you need a neural network when you have the Hamiltonian mechanics of the system modeled? I've always understood Langrangian/Hamiltonian mechanics to be methods of modeling the behavior of a system through the decomposition of the external constraints and forces acting on a body. In other words you can understand a complex model by doing some calculus on the less complex constituents of the model. I'm probably…

I love your comment. I follow, and appreciate it, but I could not help but think of this xkcd: https://xkcd.com/793/

Re: Teaching physics to neural networks removes 'chaos blindness'

#63

Post by dkersten »

esperent wrote: post #58
yongjik wrote: post #38

Earlier quoted context omitted.

Eh, I've seen my baby sucking on its thumb, and then the baby jerks and the thumb goes away, baby cries, somehow the wild flailing gets the thumb back to the mouth. Baby happy again, sucking on its own thumb. I agree that we are not born a blank slate, but at the same time, there's a lot of knowledge missing on a newborn baby.

Even if the baby has trouble getting the thumb into the mouth at first, it has no trouble sucking it. Or any of thousands of other instinctive behaviors.

Yep, the baby knows how to cry when not sucking it’s thumb for example, and not cry when its content.

Re: Teaching physics to neural networks removes 'chaos blindness'

#64

Post by jpxw »

mtgp1000 wrote: post #25
Fricken wrote: post #15

Earlier quoted context omitted.

Don't sell biology short like that. Human driver are born with a mental model of physics that's been honed 24 hours a day since before they were diatoms.

I don't think that's quite right. I believe that humans are essentially born as blank neural networks; it's the structure, and the graph of connections between brain structures and sensory inputs, that is effectively primed for learning certain tasks that we find to be intuitive. A baby is not born with the knowledge of body movement, for example, but through natural exploration of the body and environment, almost al…

This doesn’t even make sense. There must be some in-built programming on being able to learn new things, if nothing else.

Re: Teaching physics to neural networks removes 'chaos blindness'

#65

Post by nefasti »

_iyig wrote: post #56

Brings to mind this classic from the Jargon File: http://www.catb.org/~esr/jargon/html/koans.html In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. “What are you doing?”, asked Minsky. “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied. “Why is the net wired randomly?”, asked Minsky. “I do not want it to have any preconceptions of how to play”,…

I don’t get it :(

Re: Teaching physics to neural networks removes 'chaos blindness'

#66

Post by epaga »

nefasti wrote: post #65
_iyig wrote: post #56

Brings to mind this classic from the Jargon File: http://www.catb.org/~esr/jargon/html/koans.html In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. “What are you doing?”, asked Minsky. “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied. “Why is the net wired randomly?”, asked Minsky. “I do not want it to have any preconceptions of how to play”,…

I don’t get it :(

I think it means - just as closing your eyes doesn't mean the room becomes empty, wiring the learning network randomly doesn't mean you'll end up with no pre-conceptions (e.g. the rule system at least will need to be programmed in).

Re: Teaching physics to neural networks removes 'chaos blindness'

#67

Post by jojobas »

logicslave wrote: post #40
Fricken wrote: post #15

Earlier quoted context omitted.

Don't sell biology short like that. Human driver are born with a mental model of physics that's been honed 24 hours a day since before they were diatoms.

But were all a blank slate!!!

The slate structure and even size is significantly different.

Re: Teaching physics to neural networks removes 'chaos blindness'

#68

Post by taneq »

redisman wrote: post #30
mtgp1000 wrote: post #25

Earlier quoted context omitted.

I don't think that's quite right. I believe that humans are essentially born as blank neural networks; it's the structure, and the graph of connections between brain structures and sensory inputs, that is effectively primed for learning certain tasks that we find to be intuitive. A baby is not born with the knowledge of body movement, for example, but through natural exploration of the body and environment, almost al…

> A baby is not born with the knowledge of body movement Anyone who's witnessed a birth can tell you this is wrong.

They're capable of movement, I wouldn't say they have knowledge of it though. It can take them a while to even figure out suckling, and deliberate directed movements can take a month or two. Until then they're pretty much flailing randomly and gathering training data.

Re: Teaching physics to neural networks removes 'chaos blindness'

#69

Post by thesz »

Why not shamelessly plug my work here? I see no reason not to.

So, here it is: https://github.com/thesz/nn/tree/master/series

A proof of concept implementation of training neural networks process where loss function is a potential energy in Lagrangian function and I even incorporated "speed of light" - the "mass" of particle gets corrected using Lorenz multiplier m=m0/sqrt(1-v^2/c^2).

Everything is done using ideas from quite interesting paper about power of lazy semantics: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.32....

PS Proof-of-concept here means it is grossly inefficient, mainly due to amount of symbolic computation. Yet it works. In some cases. ;)

Re: Teaching physics to neural networks removes 'chaos blindness'

#70

Post by zone411 »

vutekst wrote: post #45
awinter-py wrote: post #42

> the NAIL team incorporated Hamiltonian structure into neural networks ML non-expert here. Is this the same as having an extra column of your input data that's a hamiltonian of the raw input? Or a kind of neuron that can compute a hamiltonian on an observation? Or something more complicated. is this like a specialized 'functional region' in a biological brain? (broca's area, cerebellum)

Also ML non-expert here. I think this is about a different kind of neuron(your 2nd suggestion). The paper another commenter linked says: Hamiltonian neural network (HNN) intakes position and momenta {q,p}, outputs the scalar function H, takes its gradient to find its position and momentum rates of change, and minimizes the loss which enforces Hamilton's equations of motion. https://journals.aps.org/pre/abstract/10.11…

I haven't used HNNs in practice but it seems that the main difference from common NNs is that the loss function incorporates gradients. It's not a new type of a neuron.
Post reply on HN