A Probabilistic Theory of Deep Learning
1–10 of 21 posts
Re: A Probabilistic Theory of Deep Learning
#256 pages! They should really reorganize this into 10-16 pages to get the basic ideas and results across.
Re: A Probabilistic Theory of Deep Learning
#356 pages! They should really reorganize this into 10-16 pages to get the basic ideas and results across.
I read it, started to doze off, forced myself to dig in, and was left with the feeling they just redefined the Constant of Integration as part of their own mystery lingo. It shouldn't be like this. Sheesh.
Re: A Probabilistic Theory of Deep Learning
#4I am well versed in the usual theories of learning (PAC, SQ-learning, learning with membership and equivalence queries, etc.)
Can anyone comment on how this relates to those standard models? There does not appear to be any mention in the paper of the standard learning models, and as a result I'm inclined to think this paper is not worth reading.
Re: A Probabilistic Theory of Deep Learning
#5I am well versed in the usual theories of learning (PAC, SQ-learning, learning with membership and equivalence queries, etc.) Can anyone comment on how this relates to those standard models? There does not appear to be any mention in the paper of the standard learning models, and as a result I'm inclined to think this paper is not worth reading.
On a veeeery quick skim I think it's a Bayesian generative model for deep learning architectures. I thought Zoubin Ghahramani's group already had done some similar work, but :shrug: it's not my field.
Re: A Probabilistic Theory of Deep Learning
#6What is the hypothesis? How was the hypothesis tested?
Re: A Probabilistic Theory of Deep Learning
#756 pages! They should really reorganize this into 10-16 pages to get the basic ideas and results across.
There's a place for brevity, but not in this paper. You are assuming somehow that there is redundancy in the paper. There doesn't seem to be, at first glance. The scope of the paper is large enough to merit the length.
Re: A Probabilistic Theory of Deep Learning
#8I am well versed in the usual theories of learning (PAC, SQ-learning, learning with membership and equivalence queries, etc.) Can anyone comment on how this relates to those standard models? There does not appear to be any mention in the paper of the standard learning models, and as a result I'm inclined to think this paper is not worth reading.
On a veeeery quick skim I think it's a Bayesian generative model for deep learning architectures. I thought Zoubin Ghahramani's group already had done some similar work, but :shrug: it's not my field.
To clarify, it has been studied in Zoubin Ghahramani's group [1] (and also more recently in Ryan Adam's group [2]), and it's most widely known through Radford Neal [3] who's won a lot of competitions using the Bayesian approach to NNs.
Re: A Probabilistic Theory of Deep Learning
#956 pages! They should really reorganize this into 10-16 pages to get the basic ideas and results across.
There's a place for brevity, but not in this paper. You are assuming somehow that there is redundancy in the paper. There doesn't seem to be, at first glance. The scope of the paper is large enough to merit the length.
I'm not assuming redundancy, i'm assuming what every academic does - that the author(s) should be able to concisely and accurately present their work. There's plenty of examples where complex ideas and derivations can make the journal page limit (which does vary, but i think 16 pages is good), no reason this one can't as well.
Re: A Probabilistic Theory of Deep Learning
#10What is the hypothesis? How was the hypothesis tested?
From my reading of the first twenty or so pages, it appears to be a theory of how neural networks model images (the running example). The authors claim that popular neural network architectures can be reduced to particular cases of their model.
If this model can provide an explanation for the small noises impacting NN performance on images (from karpathy.github.io, posted to HN earlier today) then that would be rocking.
Nonetheless, it does not appear to be an experimental paper, rather providing a mathematical theory of some particular classification problems.