Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teaching physics to neural networks removes 'chaos blindness'

phys.org

Post reply on HN
Print view View original

21–30 of 81 posts

Re: Teaching physics to neural networks removes 'chaos blindness'

#21

Post by vajrabum »

I believe this refers to work presented in this journal article. https://journals.aps.org/pre/abstract/10.1103/PhysRevE.101.0...

Abstract: Artificial neural networks are universal function approximators. They can forecast dynamics, but they may need impractically many neurons to do so, especially if the dynamics is chaotic. We use neural networks that incorporate Hamiltonian dynamics to efficiently learn phase space orbits even as nonlinear systems transition from order to chaos. We demonstrate Hamiltonian neural networks on a widely used dynamics benchmark, the Hénon-Heiles potential, and on nonperturbative dynamical billiards. We introspect to elucidate the Hamiltonian neural network forecasting.

Re: Teaching physics to neural networks removes 'chaos blindness'

#22

Post by gajomi »

mywittyname wrote: post #9

Why do you need a neural network when you have the Hamiltonian mechanics of the system modeled? I've always understood Langrangian/Hamiltonian mechanics to be methods of modeling the behavior of a system through the decomposition of the external constraints and forces acting on a body. In other words you can understand a complex model by doing some calculus on the less complex constituents of the model. I'm probably…

> it sounds like they've increased the accuracy of a neural network model of a system, notably for edge cases, by training it on complete a complete model of said system.

Not quite. It's really just that they require the dynamics to be Hamiltonian, which would be highly atypical of the kind of dynamics an otherwise unconstrained neural network would learn. This is reflected in their loss function, the first of which learn an arbitrary second order differential equation, the second of which enforces Hamiltonian dynamics.

I don't understand how this was considered novel enough to warrant at PRE paper.

Here is a link to the paper:

https://journals.aps.org/pre/pdf/10.1103/PhysRevE.101.062207

Re: Teaching physics to neural networks removes 'chaos blindness'

#23

Post by mhh__ »

solotronics wrote: post #19
mhh__ wrote: post #7

Earlier quoted context omitted.

Vehicle dynamics is a fairly accurate science these days (50/50 for the tires)

Racing teams and big car manufacturers have incredibly accurate models of vehicle dynamics.

But not outside those teams. If you want to put something together in a few weeks your options are relatively limited in that collecting accurate data is fairly hard. The actual dynamics of the car is fairly simple but the forces applied to it are quite hard to model (I don't know how much Michelin charge to use TameTire but I'm guessing not cheap)

Re: Teaching physics to neural networks removes 'chaos blindness'

#24

Post by cmehdy »

This sounds like the opposite of what Richard Sutton seemed to advocate for in his "Bitter Lesson"[0]. I don't know nearly enough to advocate for one thing or the other, but it is fascinating to see that those approaches seem to compete as we venture into the unknown.

[0] http://incompleteideas.net/IncIdeas/BitterLesson.html

Re: Teaching physics to neural networks removes 'chaos blindness'

#25

Post by mtgp1000 »

Fricken wrote: post #15
keenmaster wrote: post #2

I’ve said this before, but I think that a lack of physical modeling might be the key barrier for AV technology. Human drivers have a mental model of physics that they’ve honed for 17-18 hours a day since they were born.

Don't sell biology short like that. Human driver are born with a mental model of physics that's been honed 24 hours a day since before they were diatoms.

I don't think that's quite right. I believe that humans are essentially born as blank neural networks; it's the structure, and the graph of connections between brain structures and sensory inputs, that is effectively primed for learning certain tasks that we find to be intuitive.

A baby is not born with the knowledge of body movement, for example, but through natural exploration of the body and environment, almost all physically capable humans learn to walk.

Re: Teaching physics to neural networks removes 'chaos blindness'

#26

Post by fizixer »

cmehdy wrote: post #24

This sounds like the opposite of what Richard Sutton seemed to advocate for in his "Bitter Lesson"[0]. I don't know nearly enough to advocate for one thing or the other, but it is fascinating to see that those approaches seem to compete as we venture into the unknown. [0] http://incompleteideas.net/IncIdeas/BitterLesson.html

They're not the opposite, and both are correct.

Sutton is saying 'over a slightly longer time'.

You can wait 20 more years and super-duper-deep-NN-on-steroids, and hardware a million times as big and powerful, would rediscover all of theoretical physics.

Or you could inject some theoretical physics acquired by humans and make DNNs smarter today.

Re: Teaching physics to neural networks removes 'chaos blindness'

#27

Post by MereInterest »

keenmaster wrote: post #2

I’ve said this before, but I think that a lack of physical modeling might be the key barrier for AV technology. Human drivers have a mental model of physics that they’ve honed for 17-18 hours a day since they were born.

Side note, is "AV" to mean "autonomous vehicles" (assumed from context) a common usage? I've only ever heard it mean "audio/visual".

Re: Teaching physics to neural networks removes 'chaos blindness'

#28

Post by jariel »

Can someone with AI knowledge please clarify - does this mean we can build 'rules based systems' into AI to synthesise intelligence from both domains?

If so, this would be dramatic, no?

If you could teach a translation service 'grammar' and then also leverage the pattern matching, could this be a 'fundamental' new idea in AI application?

Or is this just something specific?

Re: Teaching physics to neural networks removes 'chaos blindness'

#29

Post by keenmaster »

MereInterest wrote: post #27
keenmaster wrote: post #2

I’ve said this before, but I think that a lack of physical modeling might be the key barrier for AV technology. Human drivers have a mental model of physics that they’ve honed for 17-18 hours a day since they were born.

Side note, is "AV" to mean "autonomous vehicles" (assumed from context) a common usage? I've only ever heard it mean "audio/visual".

Yes. That usage of the acronym works when there are enough context clues. I think it will supplant the "audio/visual" meaning as autonomous vehicles become more salient. Here is text from an old job posting at Ford quoted on TechCrunch:

"We are seeking exceptional candidates to join our growing Autonomous Vehicle (AV) business team!"

https://techcrunch.com/2019/03/13/ford-is-expanding-its-self...

Re: Teaching physics to neural networks removes 'chaos blindness'

#30

Post by redisman »

mtgp1000 wrote: post #25
Fricken wrote: post #15

Earlier quoted context omitted.

Don't sell biology short like that. Human driver are born with a mental model of physics that's been honed 24 hours a day since before they were diatoms.

I don't think that's quite right. I believe that humans are essentially born as blank neural networks; it's the structure, and the graph of connections between brain structures and sensory inputs, that is effectively primed for learning certain tasks that we find to be intuitive. A baby is not born with the knowledge of body movement, for example, but through natural exploration of the body and environment, almost al…

> A baby is not born with the knowledge of body movement

Anyone who's witnessed a birth can tell you this is wrong.

Post reply on HN