Large Scale Deep Learning – Jeff Dean [pdf] • Hacker News Board

Re: Large Scale Deep Learning – Jeff Dean [pdf]

#41

Post by nl » Mon, Dec 08, 2014, 6:57 PM UTC

Strilanc wrote: post #34

nl wrote: post #8

This whole slide deck is worth reading. A couple of highlights: Pg 26, quote: "Anything humans can do in 0.1 sec, the right big 10-layer network can do too". That is a very bold claim. It encompasses the entire fields of image and voice recognition as well as knowledge encoding. It's slowly becoming clear that this is likely to be true. Pg 39, 40: Google's ImageNet-winning system in 2011 had 7 layers and an error rat…

Whelp, Page 54 blew my mind right out the window. > E(hotter) - E(hot) + E(big) ≈ E(bigger) > E(Rome) - E(Italy) + E(Germany) ≈ E(Berlin) These things are linearly separable?!

Yes. Hence 'miraculous' Word2vec.

Re: Large Scale Deep Learning – Jeff Dean [pdf]

#42

Post by discardorama » Mon, Dec 08, 2014, 7:13 PM UTC

olavgg wrote: post #37

discardorama wrote: post #22

Earlier quoted context omitted.

"Used by Facebook and Google" ? Citation needed. AFAIK, at least Google has an internal homebrew solution, that automatically scales to large clusters.

AMA with Yann LeCun which confirms they are using it. http://www.reddit.com/r/MachineLearning/comments/25lnbt/ama_...

I stand corrected. Thanks for the link. I had somehow missed that AMA; it's fascinating reading. (Hinton's too).

Re: Large Scale Deep Learning – Jeff Dean [pdf]

#43

Post by ivansavz » Tue, Dec 09, 2014, 3:42 AM UTC

HN readers in the Montreal area will have a chance to listen to the talk in person at the McGill colloquium:

Scaling Deep Learning, Wednesday, December 10th, 2:00PM-3:00 PM at the McGill University M1 amphitheater of the Strathcona building at 3640 University Street.

Re: Large Scale Deep Learning – Jeff Dean [pdf]

#44

Post by fuelfive » Tue, Dec 09, 2014, 6:44 PM UTC

nl wrote: post #8

This whole slide deck is worth reading. A couple of highlights: Pg 26, quote: "Anything humans can do in 0.1 sec, the right big 10-layer network can do too". That is a very bold claim. It encompasses the entire fields of image and voice recognition as well as knowledge encoding. It's slowly becoming clear that this is likely to be true. Pg 39, 40: Google's ImageNet-winning system in 2011 had 7 layers and an error rat…

These are great highlights, although your second point from [1] is a bit misleading. Humans have a 5% error rate because the average person can't differentiate between 150 different breeds of dogs and other similar nonsense.