Why Deep Learning Works II: the Renormalization Group
charlesmartin14.wordpress.com
Why Deep Learning Works II: the Renormalization Group
1–10 of 37 posts
Re: Why Deep Learning Works II: the Renormalization Group
#2Note: This is about unsupervised learning and mostly about RBMs/DBNs. Most of the Deep Learning success is all about supervised learning. In the past, RBMs have been used for unsupervised pretraining of the model, however, nowadays, everyone uses supervised pretraining.
And the famous DeepMind works (Atari games etc) is mostly about Reinforcement learning, which is again different.
Re: Why Deep Learning Works II: the Renormalization Group
#3Note: This is about unsupervised learning and mostly about RBMs/DBNs. Most of the Deep Learning success is all about supervised learning. In the past, RBMs have been used for unsupervised pretraining of the model, however, nowadays, everyone uses supervised pretraining. And the famous DeepMind works (Atari games etc) is mostly about Reinforcement learning, which is again different.
Well, if I understood correctly, the RL DeepMind implementation is basically making a RL algorithm work with a supervised model.
Re: Why Deep Learning Works II: the Renormalization Group
#4Is the "group" in renormalization group the same "group" in group theory?
Re: Why Deep Learning Works II: the Renormalization Group
#5Is the "group" in renormalization group the same "group" in group theory?
Almost. The name "group" in renormalization group was inspired by the groups in group theory, but in reality the renormalization "group" isn't a group but a semi-group. A semi-group satisfies the same axioms as a group except for the existence of inverse. And that is the mathematical reason why you can describe big things in terms of smaller things, but you can't describe small things in terms of bigger things: the renormalization (semi-) group flows from the ultraviolet to the infrared, but not the other way around :-)
Re: Why Deep Learning Works II: the Renormalization Group
#6Okay, I confess. I really didn't understand most of that post. It sounds really smart, but someone will have to vouch that it's legit, because the picture of Kadanoff cuddling Cookie Monster trigged my baloney detector https://charlesmartin14.files.wordpress.com/2015/04/kadanoff...
Re: Why Deep Learning Works II: the Renormalization Group
#7Okay, I confess. I really didn't understand most of that post. It sounds really smart, but someone will have to vouch that it's legit, because the picture of Kadanoff cuddling Cookie Monster trigged my baloney detector https://charlesmartin14.files.wordpress.com/2015/04/kadanoff...
I will read the post later, but I can tell you right now that I'm a physicist and when I was first introduced to deep learning my first thought, especially in the context of visual recognition, was "this smells of renormalization group".
Re: Why Deep Learning Works II: the Renormalization Group
#8Is the "group" in renormalization group the same "group" in group theory?
oneloop, your comment looks helpful but you are hell-banned, you may want to email HN to be reinstated.
--
Edit - he's been reinstated.
Re: Why Deep Learning Works II: the Renormalization Group
#9Is the "group" in renormalization group the same "group" in group theory?
Kinda. The 'group' refers to a group scale and/or conformal transformations. In the context of Deep Learning, the 'scale' transform is akin to adding layers.
Re: Why Deep Learning Works II: the Renormalization Group
#10Okay, I confess. I really didn't understand most of that post. It sounds really smart, but someone will have to vouch that it's legit, because the picture of Kadanoff cuddling Cookie Monster trigged my baloney detector https://charlesmartin14.files.wordpress.com/2015/04/kadanoff...
I try to lighten things up because the math by be unfamiliar.