No post body was provided.
Untitled topic
1–10 of 15 posts
Re: undefined
#2ELI5?
Re: undefined
#3ELI5?
Conventional probability theory, as commonly applied, seems to have some major blind spots. Using deep learning, Yann LeCun's team was able to find uniquely different sets of points that had the same statistical properties. They are calling into question the approach of throwing probability theory at every and any problem. Nonlinear systems in particular often exhibit behavior that is not easy to grasp via probability theory.
Re: undefined
#4That gif is wildly intriguing to me. Probability theory is the bread and butter of mainstream science. Have people been doing it wrong this whole time?
I want to dig deeper into this but two things come to mind:
1. George Sugihara's and Floris Takens' works on quantifying causation in non-linear systems. Correlation doesn't imply causation, so why not look at causation? Here's a writeup on it: https://www.quantamagazine.org/chaos-theory-in-ecology-predi...
2. Andrei Khrennikov argues that quantum models can be expressed with Kolmogorovian (classical) probability theory. He points out that many probability theorists have glossed over an entire subtlety of Kolmogorov's work: that probability spaces need not be finite dimensional, that they can be subspaces of higher dimensional, even infinite dimensional probability spaces. Khrennikov's philosophical convictions are not widely shared by quantum theorists, but his math is solid (he seems to be in pursuit of quantum theories with "hidden variables", like Einstein was looking for). http://www.sciencedirect.com/science/article/pii/S0375960103...
Even in the gif shown in this article, you might suppose that each point set could be modeled in higher dimensional terms, for example adding some kind of discrete curl-like measure. But even without going there: something as simple as a changing to a polar basis should reveal different statistics.
And isn't this kind of what deep learning is all about? Going between higher and lower dimensional representations?
Re: undefined
#5That gif is wildly intriguing to me. Probability theory is the bread and butter of mainstream science. Have people been doing it wrong this whole time? I want to dig deeper into this but two things come to mind: 1. George Sugihara's and Floris Takens' works on quantifying causation in non-linear systems. Correlation doesn't imply causation, so why not look at causation? Here's a writeup on it: https://www.quantamagaz…
I'm not sure why that gif is so interesting to you; that wasn't the reason I submitted this piece. Average and standard deviation are hardly useful data points in evaluating a dataset, and also don't have anything to do with probability theory. The second half of the post contains the bulk of its interesting content (as I read it).
Re: undefined
#6That gif is wildly intriguing to me. Probability theory is the bread and butter of mainstream science. Have people been doing it wrong this whole time? I want to dig deeper into this but two things come to mind: 1. George Sugihara's and Floris Takens' works on quantifying causation in non-linear systems. Correlation doesn't imply causation, so why not look at causation? Here's a writeup on it: https://www.quantamagaz…
For those who want the source of the gif : https://www.autodeskresearch.com/publications/samestats
The paper is actually really interesting, as it provides a useful insight into what summary statistics mean and the importance of data visualization.
Re: undefined
#7As someone who has seen enough of the math, I strongly feel that probability just makes ideas seem "sexier" in ML; often you don't really need probability, or you only need it to use Jensen. Worse, using probabilities generally masks deeper ideas from surfacing. Bayesian-ism, has become another cargo-cult, in many ways.
Re: undefined
#8So the author quotes Judea Pearl, whose causal calculus led to PGMs playing a critical in computer science and science as a whole, and somehow uses that to support the idea that probability theory is useless? That is absurd and out of context.
Take a look at Kaldi or any state of the art platform for sound to speech and NLP analysis, or take a look at deep belief machines/networks. They all roughly extend probabilistic graphical models with deep architectures and some nonlinearity to do what couldn't be done before. (See also the Neural Hawkes Process [https://arxiv.org/abs/1612.09328].)
The article seems to primarily be about promoting his book instead of having a cohesive argument.
Yes, nonlinearity matters, but I'm not remotely convinced that tossing probability is a good thing.
Re: undefined
#9ELI5?
Conventional probability theory, as commonly applied, seems to have some major blind spots. Using deep learning, Yann LeCun's team was able to find uniquely different sets of points that had the same statistical properties. They are calling into question the approach of throwing probability theory at every and any problem. Nonlinear systems in particular often exhibit behavior that is not easy to grasp via probabilit…
But deep learning is terrible at learning nonlinear systems which are easy to model with things like stochastic differential equations... this is just another case of machine learning being pushed too far.
Re: undefined
#10ELI5?
There is no silver bullet.
Things work until they don't.