Why Deep Learning Works II: the Renormalization Group • Hacker News Board

Re: Why Deep Learning Works II: the Renormalization Group

#31

Post by selimthegrim » Sun, Jul 05, 2015, 10:11 PM UTC

contravariant wrote: post #27

oneloop wrote: post #7

Earlier quoted context omitted.

I will read the post later, but I can tell you right now that I'm a physicist and when I was first introduced to deep learning my first thought, especially in the context of visual recognition, was "this smells of renormalization group".

Do you happen to know any good introductory texts to the renormalization group?

The standard one in use at Santa Barbara is http://www.amazon.com/Lectures-Transitions-Renormalization-F...

Some others I have sitting beside me at my desk right now are http://www.amazon.com/Theory-Critical-Phenomena-Introduction..., http://www.amazon.com/Renormalization-Introduction-Operator-... and http://www.amazon.com/Renormalization-Methods-Guide-For-Begi...

The most modern treatment is probably http://www.amazon.com/Scaling-Renormalization-Statistical-Ph...

(For God's sake, stay away from anything written by Zinn-Justin, Itzykson, or Zuber unless you know what you're doing)

Re: Why Deep Learning Works II: the Renormalization Group

#32

Post by [deleted] » Sun, Jul 05, 2015, 11:27 PM UTC

sgt101 wrote: post #30

I don't like the assertion at all because so many techniques are held to be "deep learning" and because even when specific techniques are built on an analogy of this sort (think Simulated Annealing and Genetic Algorithms) they do not work "because" they are "like" the physical processes that served as an inspiration. Names are useful, but only as a aide to thinking. Does this help us think about these techniques?

[deleted]

Re: Why Deep Learning Works II: the Renormalization Group

#33

Post by octatoan » Mon, Jul 06, 2015, 2:37 AM UTC

No MathJax. I am disappoint.

Re: Why Deep Learning Works II: the Renormalization Group

#34

Post by abecedarius » Mon, Jul 06, 2015, 2:48 AM UTC

curiousjorge wrote: post #29

It always depresses me when I read anything with math formulas and esoteric terms, a constant reminder of my lifelong incompetence with math and university calculus courses.

I expect very few people learned from this post; I didn't, and I kinda like math. This was the sort of math writing that makes sense only given nearly the same background as the author. (Someone above posted specific complaints about the unclear notation.)

Re: Why Deep Learning Works II: the Renormalization Group

#35

Post by fizixer » Mon, Jul 06, 2015, 4:30 AM UTC

One way of think of it is that:

There are connections between Deep Learning and Theoretical Physics because there are (even stronger) connections between Information Theory and Statistical Mechanics.

Re: Why Deep Learning Works II: the Renormalization Group

#36

Post by ElectricFeel » Mon, Jul 06, 2015, 4:45 AM UTC

incredibly impressive research article you've posted!

Re: Why Deep Learning Works II: the Renormalization Group

#37

Post by make3 » Mon, Jul 06, 2015, 5:58 AM UTC

people don't use unsupervised pre training in deep learning anymore afaik