even after cross-validation?
Machine Learning’s ‘Amazing’ Ability to Predict Chaos
11–20 of 77 posts
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#12Novel paper. But it seems like a lot of the excitement is because of the fusion of two buzz-words, one from the 80s and 90s and another from the 2010s. So, the output is a bunch of stuff coming out of a kinda simple dynamical system. Chaotic for sure, but still simple. Deep learning (and more generally, recurrent neural nets, LSTMs, and derivatives thereof) has been shown capable of learning much more complex nonlinear systems, including human perception. Given this, I think the OP paper is a low-hanging fruit. It was only a matter of time before someone figured out a way to learn specific chaotic nonlinear dynamical systems. Nice work nonetheless.
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#13[deleted]
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#14Novel paper. But it seems like a lot of the excitement is because of the fusion of two buzz-words, one from the 80s and 90s and another from the 2010s. So, the output is a bunch of stuff coming out of a kinda simple dynamical system. Chaotic for sure, but still simple. Deep learning (and more generally, recurrent neural nets, LSTMs, and derivatives thereof) has been shown capable of learning much more complex nonline…
I'm not sure if human perception is a chaotic system.
Chaos is defined as "small change in input -> large change in output".
Perception is actually the opposite, with small input changes (changes in light, different angles,...) leading to fundamentally unchanged perception ("It's a tree").
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#15Novel paper. But it seems like a lot of the excitement is because of the fusion of two buzz-words, one from the 80s and 90s and another from the 2010s. So, the output is a bunch of stuff coming out of a kinda simple dynamical system. Chaotic for sure, but still simple. Deep learning (and more generally, recurrent neural nets, LSTMs, and derivatives thereof) has been shown capable of learning much more complex nonline…
That's like saying, "it was only a matter of time someone figured out time traveling", when it happens. The whole point of the investigation was about chaotic systems, regardless of whether they're complex or not.
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#16Isn't deterministic-chaos an oxymoron?
The specific kind of chaos here implies (roughly) that a system starting in state X and a system starting arbitrarily close to X will at some point in the future behave totally differently. Not different as in "they'll drift further apart," but different as in "this one rolls off the cliff and this doesn't."
It's a fascinating field of dynamical systems, which is one of the coolest subfields of math (since the world is dynamic). Here's the wikipedia page https://en.wikipedia.org/wiki/Chaos_theory
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#17Don't be too seduced by the enticing ideas at the end of the article. The disconnect here is that success in learning how to predict the results of an algorithmic simulation is not really indicative of how it would perform with the decidedly non-algorithmic natural behavior of weather or earthquakes, phenomena which don't operate in a closed system with predefined limits and parameters. It sounds like the next step,…
(Part of) the idea seems to be synthesizing missing data from later observations of the known data, i. e. "to arrive at this state (A_t,C_t,E_t), the initial state must have been...(A_t-1,B_t-1,C_t-1,E_t-1)..."
I have my doubts that this can just overcome the fundamental problem of chaos, but it doesn't sound impossible.
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#18Novel paper. But it seems like a lot of the excitement is because of the fusion of two buzz-words, one from the 80s and 90s and another from the 2010s. So, the output is a bunch of stuff coming out of a kinda simple dynamical system. Chaotic for sure, but still simple. Deep learning (and more generally, recurrent neural nets, LSTMs, and derivatives thereof) has been shown capable of learning much more complex nonline…
In the example they give they predict the future of a simulation. If you have a perfect simulation, with perfect observation, why not just run the simulation forwards? Well, the goal is to apply this to the real world, where you possibly have only an approximate model and observations - which are noisy and imprecise. So, instead, try predicting the future based on noisy observations. Due to exponential divergence, it seems unlikely this would work. Looking through the paper, it looks like they do not analyze the performance under noisy observation - they just analyze their ability to estimate the Lyapunov exponents under noise, which is much easier.
So the real world application (in terms of forward forecasting) seems like it's limited to cases where chaotic divergence between simulation and real world is due to simulation model errors rather than observation error - the latter is still a fundamental limit. Otherwise, this demonstrates that DNNs can be trained to solve diff eqs well, which is fairly well-trodden work.
As a result, claiming that this can enhance ex. weather prediction is highly misleading at this point, as nothing has demonstrated any prediction performance improvements under noisy observation, which is really the fundamental (and practical) limit in chaotic systems. While it still may be possible to use DNNs to do that (by ex. learning how to most accurately estimate the components of state with the largest Lyapunov constants) I don't think this work demonstrates the feasibility of that yet. The language in the paper is more reserved, in that they primarily claim that they can use the DNN to learn a dynamic model even with observation noise and then that model can be used to estimate Lyapunov constants accurately. That's much more reasonable.
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#19Can we have more publishing like Quanta magazine? It's just perfect in terms of reporting from the frontier, being easy to unsetstand, and not talking down to you.
It's funded by Jim Simons, of the hedge fund Renaissance Technologies. A mathematician with money, trying to promote mathematics. Quanta magazine is a fantastic service.
But the other Renaissance magnate, Bob Mercer, as his own media hobby funded Breitbart News.
Re: Machine Learning’s ‘Amazing’ Ability to Predict Chaos
#20Novel paper. But it seems like a lot of the excitement is because of the fusion of two buzz-words, one from the 80s and 90s and another from the 2010s. So, the output is a bunch of stuff coming out of a kinda simple dynamical system. Chaotic for sure, but still simple. Deep learning (and more generally, recurrent neural nets, LSTMs, and derivatives thereof) has been shown capable of learning much more complex nonline…
That's like saying, "it was only a matter of time someone figured out time traveling", when it happens. The whole point of the investigation was about chaotic systems, regardless of whether they're complex or not.
Nope, given our current theories time travel is practically impossible. On the other hand we know that chaotic systems are functions and that matrices can approximate any function.