Earlier quoted context omitted.
Is this a meme? No they have not. Sure they are a popular hardware provider, but NN have been around before they were even a company. I think Fukushima deserves a spot here too, before giving it to Nvidia, but there are so many people that have contributed to NN, so I guess those in the spotlight take the glory if it must be taken.
...Fukushima? Is it a typo for someone (or some company)?
Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
91–100 of 159 posts
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#92This is well deserved, but IMO it also signals an upcoming AI winter. This cycle has happened several times before: 1) Some scrappy researcher on a low budget develops a new AI technology that shows promise in a specific area. 2) More researchers take that idea and successfully apply it more broadly. 3) Massive investment in research happens, pushing PhD candidates into extracting every possible nuance out of that te…
The assumption here is that 6) "can't be used broadly" is true. What evidence do you have for this? It turns out this technology is already being used everywhere: text translation, image recognition, autonomous vehicles, text generation and so on. It's already generating huge value which is why so many companies are investing so much in it. There isn't going to be any Winter coming anytime soon.
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#93This is well deserved, but IMO it also signals an upcoming AI winter. This cycle has happened several times before: 1) Some scrappy researcher on a low budget develops a new AI technology that shows promise in a specific area. 2) More researchers take that idea and successfully apply it more broadly. 3) Massive investment in research happens, pushing PhD candidates into extracting every possible nuance out of that te…
There's good reason to think that the AI Winter / AI Spring cycle is done. AI techniques are now creating so much real world value (more so than in the 80's / 90's) that there may well not be any more "AI Winter" events. Or perhaps they will be, but they'll be less pronounced. Maybe an "AI Fall" instead of "AI Winter". Part of that too, is that I think people now realize that narrow AI is sufficient to create tremend…
AI winter doesn't mean people stop using it. AI winter and spring aren't a phenomenon specific to AI...in essence it's the same dynamic found in economic bubbles. In almost every economic bubble we see actual economic benefit underlying the hype, but the hype grows bigger than the benefit can account for, triggering eventual collapse of the hype. The thing that you should notice is that when the bubble collapses we don't stop buying, we just stop overinvesting.
We haven't stopped buying tulips, trains, stocks, technology, or real estate. And just as well, we haven't stopped using symbolic AI, single-layer neural nets, ensemble models, expert systems, or logic programming languages. The new topological enhancements to Neural Nets won't ever go away either...but that doesn't mean we won't see a drop in investment once the general public realizes that your neural nets aren't going to learn how to do double-entry accounting any time soon. The AI winter isn't characterized by the technology going away, it is characterized by lofty idealism being shattered and investment dropping back to reflect reality.
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#94I think Schmidhuber should be up there. LSTMs are everywhere, not just NLP. The latest Starcraft 2 bot from Deepmind uses LSTMs extensively in its architecture.
Schmidhuber's significance in the field is not at the same level. He invented one type of neural network, but the "Canadian mafia" launched the whole Deep Learning revolution with multiple breakthroughs and theoretical understanding. LSTM's are clever and they were bleeding edge up to 2014, but once they were understood better as a bypass mechanism, attention, context vectors and averaging networks and causal convolu…
That's very ignorant to say that attention or convolution is anywhere close to the expressivity of RNNs(LSTM). Instead of reading a few cherry-picked results from a model with extremely tuned hyperparameters, pick any random set of tasks and experiment yourself. Based on my experience, LSTM always performs better than all unless you brutely search over hundreds of hyperparameters configs.
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#95It saddens me to see these people being given awards for improving the surveillance and manipulation capabilities of advertising corporations. This isn't how you encourage people to use their abilities to contribute to society in a positive way.
Let’s withdraw the Nobel from Einstein while we’re at it.
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#96For a more balanced view of the history of deep learning see Schmidhuber‘s review http://people.idsia.ch/~juergen/deep-learning-overview.html and http://people.idsia.ch/~juergen/who-invented-backpropagation... .
Direct link with the pdf version: http://people.idsia.ch/~juergen/DeepLearning2July2014.pdf
Unfortunately, a good PR is definitely important for prizes.
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#97I think Schmidhuber should be up there. LSTMs are everywhere, not just NLP. The latest Starcraft 2 bot from Deepmind uses LSTMs extensively in its architecture.
Schmidhuber's significance in the field is not at the same level. He invented one type of neural network, but the "Canadian mafia" launched the whole Deep Learning revolution with multiple breakthroughs and theoretical understanding. LSTM's are clever and they were bleeding edge up to 2014, but once they were understood better as a bypass mechanism, attention, context vectors and averaging networks and causal convolu…
"Schmidhuber's significance in the field is not at the same level"
Out of curiosity, at the same level as any of the other three? I could see it from Hinton, maybe LeCunn, but Bengio?
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#98This is well deserved, but IMO it also signals an upcoming AI winter. This cycle has happened several times before: 1) Some scrappy researcher on a low budget develops a new AI technology that shows promise in a specific area. 2) More researchers take that idea and successfully apply it more broadly. 3) Massive investment in research happens, pushing PhD candidates into extracting every possible nuance out of that te…
The assumption here is that 6) "can't be used broadly" is true. What evidence do you have for this? It turns out this technology is already being used everywhere: text translation, image recognition, autonomous vehicles, text generation and so on. It's already generating huge value which is why so many companies are investing so much in it. There isn't going to be any Winter coming anytime soon.
You've identified four areas where it excels. Only a few billion more areas to go. And sure, we can now have algorithms that can categorize dogs based on collections of pixels better than humans can. Get back to me when they've figured out double entry accounting, splitting atoms, building bridges and spaceships, or improve upon TCP-IP.
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#99Earlier quoted context omitted.
The assumption here is that 6) "can't be used broadly" is true. What evidence do you have for this? It turns out this technology is already being used everywhere: text translation, image recognition, autonomous vehicles, text generation and so on. It's already generating huge value which is why so many companies are investing so much in it. There isn't going to be any Winter coming anytime soon.
You've identified four areas where it excels. Only a few billion more areas to go. And sure, we can now have algorithms that can categorize dogs based on collections of pixels better than humans can. Get back to me when they've figured out double entry accounting, splitting atoms, building bridges and spaceships, or improve upon TCP-IP.
This is a computer science award, nobody is claiming that AI will revolutionize the fields you mention (which are at best loosely related to CS). It doesn't have to be useful literally everywhere to have value.
Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award
#100Earlier quoted context omitted.
Schmidhuber's significance in the field is not at the same level. He invented one type of neural network, but the "Canadian mafia" launched the whole Deep Learning revolution with multiple breakthroughs and theoretical understanding. LSTM's are clever and they were bleeding edge up to 2014, but once they were understood better as a bypass mechanism, attention, context vectors and averaging networks and causal convolu…
That's very ignorant to say that attention or convolution is anywhere close to the expressivity of RNNs(LSTM). Instead of reading a few cherry-picked results from a model with extremely tuned hyperparameters, pick any random set of tasks and experiment yourself. Based on my experience, LSTM always performs better than all unless you brutely search over hundreds of hyperparameters configs.
> That's very ignorant to say that attention or convolution is anywhere close to the expressivity of RNNs(LSTM).
But that's exactly the point of the Transformer model, with a paper aptly titled "Attention is all you need" [1]. And the Bert architecture, based in this idea, seems to be doing well. And they claim to be bery flexible, too[2].
Maybe that's what you meant with "unless you brutely search over hundreds of hyperparameters configs", but then again, isn't that what NNs are about anyway?