Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

nytimes.com

Post reply on HN
Print view View original

111–120 of 159 posts

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#111

Post by phowon »

probably_wrong wrote: post #100
anyBodyCanDoDL wrote: post #94

Earlier quoted context omitted.

That's very ignorant to say that attention or convolution is anywhere close to the expressivity of RNNs(LSTM). Instead of reading a few cherry-picked results from a model with extremely tuned hyperparameters, pick any random set of tasks and experiment yourself. Based on my experience, LSTM always performs better than all unless you brutely search over hundreds of hyperparameters configs.

> That's very ignorant to say that attention or convolution is anywhere close to the expressivity of RNNs(LSTM). But that's exactly the point of the Transformer model, with a paper aptly titled "Attention is all you need" [1]. And the Bert architecture, based in this idea, seems to be doing well. And they claim to be bery flexible, too[2]. Maybe that's what you meant with "unless you brutely search over hundreds of h…

The success of Transformers aside, I'm not sure you should be relying on model titles for anything, lest we forget papers like "One Model To Learn Them All" [1].

[1] https://arxiv.org/abs/1706.05137

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#112

Post by lettergram »

nabla9 wrote: post #44
obviuosly wrote: post #37

Earlier quoted context omitted.

Various researchers, e.g. Werbos, Linnainmaa, Bryson and Yu-Chi Ho have been doing backprop in neural networks before Hinton et al. Hinton was merely a popularizer of the idea.

Wright brothers were most likely not the first to achieve heavier-than-air powered flight. But they made the first controlled, sustained flight. Controlled flight was the real breakthrough in aviation. It's the same with Hinton et. all. They were among many other pioneers, but what really sets them apart is that they made it all work. They also analyzed why it works.

For reference:

https://en.m.wikipedia.org/wiki/Gustave_Whitehead

Ive seen some debate around Gustave Whitehead. There apparently is some evidence that he achieved powered flight as early as 1899.

It is claimed, he did not speak English very well. So it’s suspected he just didn’t publicize his work as much.

Replicas of his early flying machine designs have been shown to fly. Including one from 1901:

https://en.m.wikipedia.org/wiki/Whitehead_No._21

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#113

Post by freyir »

orbifold wrote: post #5

For a more balanced view of the history of deep learning see Schmidhuber‘s review http://people.idsia.ch/~juergen/deep-learning-overview.html and http://people.idsia.ch/~juergen/who-invented-backpropagation... .

I could imagine this announcement will drive Schmidhuber up the wall. He's been very outspoken about how credit in the field should be assigned.

That said, I don't know how you could argue against Hinton's impact on the field, or LeCun and Bengio's achievements.

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#114

Post by mindcrime »

darksaints wrote: post #93
mindcrime wrote: post #87

Earlier quoted context omitted.

There's good reason to think that the AI Winter / AI Spring cycle is done. AI techniques are now creating so much real world value (more so than in the 80's / 90's) that there may well not be any more "AI Winter" events. Or perhaps they will be, but they'll be less pronounced. Maybe an "AI Fall" instead of "AI Winter". Part of that too, is that I think people now realize that narrow AI is sufficient to create tremend…

AI winter doesn't mean people stop using it. AI winter and spring aren't a phenomenon specific to AI...in essence it's the same dynamic found in economic bubbles. In almost every economic bubble we see actual economic benefit underlying the hype, but the hype grows bigger than the benefit can account for, triggering eventual collapse of the hype. The thing that you should notice is that when the bubble collapses we d…

AI winter doesn't mean people stop using it. ... The thing that you should notice is that when the bubble collapses we don't stop buying, we just stop overinvesting.

Right, but that doesn't match the way I feel the term "AI Winter" has been used. Now I could be mis-interpreting things, but I've always looked at an "AI Winter" as a period of underinvestment, created as an over-reaction to the mechanics you're referring to.

but that doesn't mean we won't see a drop in investment once the general public realizes that your neural nets aren't going to learn how to do double-entry accounting any time soon.

Right, but again, I don't think most people use "AI Winter" to mean a simple "drop in investment". If it were something that straightforward, there would be no need for the "Winter" metaphor.

And that's why I say there may indeed be a drop... an "AI Fall" if you will, that still represents a pull-back of sorts, but perhaps just a less pronounced and extended pullback like we've seen in the past.

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#115

Post by mindcrime »

darksaints wrote: post #103
bernardopires wrote: post #99

Earlier quoted context omitted.

This is a computer science award, nobody is claiming that AI will revolutionize the fields you mention (which are at best loosely related to CS). It doesn't have to be useful literally everywhere to have value.

Yes, people actually are claiming that these new NNet advancements are the key that will unlock AGI, which by its very nature implies that they will be as advanced, or more advanced, than humans in all forms of intelligence. They're lightyears away from that, but now the general public is buying into the hype and thinking it's a decade or two away. NNets are already a disappointment to nearly any applied researcher t…

Yes, people actually are claiming that these new NNet advancements are the key that will unlock AGI, which by its very nature implies that they will be as advanced, or more advanced, than humans in all forms of intelligence.

Who are "people" in this context? Are you talking about the general public, or misinformed journalists who are writing about things they don't understand? Because from what I've seen, by and large, the researchers working on Deep Learning are not claiming that DL (alone) is sufficient to achieve AGI, nor do they posit that AGI is anywhere close to reality. Read, for example, Martin Ford's book Architects of Intelligence[1] and note how the various researchers interviewed talk about AGI. That list, BTW, includes all three of LeCun, Hinton, and Bengio, as well as many others.

[1]: http://book.mfordfuture.com/

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#116

Post by srean »

backpropaganda wrote: post #84
obviuosly wrote: post #37

Earlier quoted context omitted.

Various researchers, e.g. Werbos, Linnainmaa, Bryson and Yu-Chi Ho have been doing backprop in neural networks before Hinton et al. Hinton was merely a popularizer of the idea.

"merely" People deserve credit for taking an idea and running with it. Without Hinton's efforts (in 1987, 2006, or 2012), deep learning would have never seen the explosive growth.

But is PR what Turing award is for though ?

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#117

Post by srean »

nabla9 wrote: post #33
li4ick wrote: post #29

I think Schmidhuber should be up there. LSTMs are everywhere, not just NLP. The latest Starcraft 2 bot from Deepmind uses LSTMs extensively in its architecture.

Schmidhuber's significance in the field is not at the same level. He invented one type of neural network, but the "Canadian mafia" launched the whole Deep Learning revolution with multiple breakthroughs and theoretical understanding. LSTM's are clever and they were bleeding edge up to 2014, but once they were understood better as a bypass mechanism, attention, context vectors and averaging networks and causal convolu…

So ... Schmidhuber's was the seminal work and the later ones were incremental improvements that came from an understanding of Schmidhuber's work ?

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#118

Post by simonster »

nabla9 wrote: post #44
obviuosly wrote: post #37

Earlier quoted context omitted.

Various researchers, e.g. Werbos, Linnainmaa, Bryson and Yu-Chi Ho have been doing backprop in neural networks before Hinton et al. Hinton was merely a popularizer of the idea.

Wright brothers were most likely not the first to achieve heavier-than-air powered flight. But they made the first controlled, sustained flight. Controlled flight was the real breakthrough in aviation. It's the same with Hinton et. all. They were among many other pioneers, but what really sets them apart is that they made it all work. They also analyzed why it works.

Schmidhuber's take on the Wright brothers:

https://www.nature.com/articles/421689c

https://www.nature.com/articles/d41586-019-00491-5

Re: Yann LeCun, Geoffrey Hinton and Yoshua Bengio win Turing Award

#119

Post by srean »

dang wrote: post #74
Nimitz14 wrote: post #35

Earlier quoted context omitted.

Schmidhuber did way more than LSTMs. And they still are SOTA on many tasks. You clearly are not very familiar with the field.

Whoa. Personal attacks will get you banned here. We've warned you about this more than once before. Please review https://news.ycombinator.com/newsguidelines.html and follow the rules when posting here. When you do, don't miss this one, since it should have changed your assumption about the GP: " Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize…

Just a note, I did not find it any more rude than other comments that get a free pass. Commenting that more familiarity with the literature is necessary can be a very factual claim.
Post reply on HN