Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

Post by ignoramous » Wed, Mar 27, 2019, 1:03 PM UTC

li4ick wrote: post #30

ignoramous wrote: post #14

Geoffrey Hinton's tech-talk in 2007 (!) at Google is a great watch (with a heavy dose of technical jargon [0] plus some dry British humour interlaced throughout). He explains digit recognition (vs SVM) [1], document classification (vs LSH), and breifly summarises image classification [2] problems and how they were solved: https://youtu.be/AyzOUbkUf3M You could instantly see the results he presents were way better tha…

Although 3b1b's videos are awesome, his NN videos are heavily inspired by this: http://neuralnetworksanddeeplearning.com/ . Even the code on his github page is taken from there.

Post by nabla9 » Wed, Mar 27, 2019, 1:20 PM UTC

li4ick wrote: post #29

I think Schmidhuber should be up there. LSTMs are everywhere, not just NLP. The latest Starcraft 2 bot from Deepmind uses LSTMs extensively in its architecture.

Post by Nimitz14 » Wed, Mar 27, 2019, 1:22 PM UTC

nabla9 wrote: post #33

li4ick wrote: post #29

I think Schmidhuber should be up there. LSTMs are everywhere, not just NLP. The latest Starcraft 2 bot from Deepmind uses LSTMs extensively in its architecture.

Schmidhuber's significance in the field is not at the same level. He invented one type of neural network, but the "Canadian mafia" launched the whole Deep Learning revolution with multiple breakthroughs and theoretical understanding. LSTM's are clever and they were bleeding edge up to 2014, but once they were understood better as a bypass mechanism, attention, context vectors and averaging networks and causal convolu…

Post by obviuosly » Wed, Mar 27, 2019, 1:36 PM UTC

bobosha wrote: post #12

orbifold wrote: post #5

For a more balanced view of the history of deep learning see Schmidhuber‘s review http://people.idsia.ch/~juergen/deep-learning-overview.html and http://people.idsia.ch/~juergen/who-invented-backpropagation... .

No invention occurs in a vacuum, most creations are recombinant effects.The pieces of the puzzle might very well have existed, but Hinton deservedly receives credit for putting the various pieces together for backprop IMHO.

Post by amelius » Wed, Mar 27, 2019, 1:44 PM UTC

nabla9 wrote: post #36

amelius wrote: post #18

I think NVidia deserves part of the award, as they have been a major factor in the progress of NNs.

They don't deserve science award because they didn't do science or pioneer anything. They just responded to demand. They have been awarded with money. It was the high performance scientific computing community that started to use graphics cards to perform matrix computations in the early 2000's. The term GPGPU term was invented. GPU programming pain in the ass in early 2000's. Nvidia saw some markets in GPGPU's for s…