Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How New Are Yann LeCun's “New” Ideas?

garymarcus.substack.com

Post reply on HN
Print view View original

71–80 of 90 posts

Re: How New Are Yann LeCun's “New” Ideas?

#71

Post by bjourne »

paganel wrote: post #64
bjourne wrote: post #49

So the idea is that statistical language modelling is not enough. You need a model based on logic too for "real" artificial intelligence. I wonder what the evidence for this claim is? Because the inferences and reasoning GPT3 is already capable of is incredible and beats most expert systems that I know of. And GPT4 is around the corner, Stable Diffusion was published like only a few months ago. I don't see why not mo…

> Stable Diffusion was published like only a few months ago Honest question: what's "intelligence"-like about Stable Diffusion?

Because being able to draw "a painting of joe biden as king kong on top of a skyscraper in the style of monet" was something that until very recently were thought of as requiring intelligence. Of course, now it is not so impressive anymore because it is all mathematics and digital logic. But that is the problem with defining artificial intelligence. Any time a task is implemented on a computer you can point to that implementation as evidence that the task didn't require intelligence after all. Many decades ago researchers thought that playing chess on a high level required intelligence, then go, then poker, then composing music, then driving a car, etc... Nowadays researchers are more cautious and don't state that "solving task X implies intelligence". Thus it becomes a moving target and a computer can never prove itself intelligent.

Re: How New Are Yann LeCun's “New” Ideas?

#72

Post by turkeygizzard »

blueyes wrote: post #6

Wow, Gary Marcus just Schmidhubered Yann LeCun. The ironic thing of course is that Yann has not been at the forefront of AI for many many years (and Gary, of course, never has). Facebook's research has failed to rival Google Brain, DeepMind, OpenAI, and groups at top universities. So to the extent that Yann is copying Gary's opinions, it's because they both converge at a point far behind the leaders in the field. Yan…

Sorry, could you explain Schmidhubering as a verb? I know who Schmidhuber is, but not familiar enough to understand this. Is it that Schmidhuber makes claims that LeCun's and others' ideas are derivative of his own?

Re: How New Are Yann LeCun's “New” Ideas?

#73

Post by minihat »

turkeygizzard wrote: post #72
blueyes wrote: post #6

Wow, Gary Marcus just Schmidhubered Yann LeCun. The ironic thing of course is that Yann has not been at the forefront of AI for many many years (and Gary, of course, never has). Facebook's research has failed to rival Google Brain, DeepMind, OpenAI, and groups at top universities. So to the extent that Yann is copying Gary's opinions, it's because they both converge at a point far behind the leaders in the field. Yan…

Sorry, could you explain Schmidhubering as a verb? I know who Schmidhuber is, but not familiar enough to understand this. Is it that Schmidhuber makes claims that LeCun's and others' ideas are derivative of his own?

Schmidhuber is a prolific flag-planter who is notorious for publicly raising a stink when he deems he should've been cited, but wasn't. It's happened enough that it's now a meme in the ML community.

Re: How New Are Yann LeCun's “New” Ideas?

#74

Post by actually_a_dog »

trrbb123 wrote: post #19
actually_a_dog wrote: post #15

Earlier quoted context omitted.

So what? Is he actually right, or is he wrong? A good argument delivered badly is still a good argument.

Name one academic you look up to who never admits he is wrong.

I'll be glad to do so after you explain how that's relevant any argument they may make.

Re: How New Are Yann LeCun's “New” Ideas?

#76

Post by trobertson »

minihat wrote: post #73
turkeygizzard wrote: post #72

Earlier quoted context omitted.

Sorry, could you explain Schmidhubering as a verb? I know who Schmidhuber is, but not familiar enough to understand this. Is it that Schmidhuber makes claims that LeCun's and others' ideas are derivative of his own?

Schmidhuber is a prolific flag-planter who is notorious for publicly raising a stink when he deems he should've been cited, but wasn't. It's happened enough that it's now a meme in the ML community.

It's important to mention that Schmidhuber is usually correct, in that his lab has been decades ahead in both theory and proofs-of-concept. The reason his lab is so under-cited is that his lab made these advances before the hardware to practically do it was available. Now that the techniques can be run, it's the people running them that tend to get all the credit for being "first".

Re: How New Are Yann LeCun's “New” Ideas?

#77

Post by mizzao »

jackblemming wrote: post #11

None of Gary’s comments were original either. I don’t know what I’d call this, but I’ve seen similar behavior elsewhere. This weird “flag planting” behavior to try to get credit without doing any actual work, as well as disregarding all prior work. Normally the “predictions” are vague or could be applied to anything. It seems borderline like a mental illness of some sort, but I’m not a mental health professional.

The problem is that ideas are the currency of academia, so you have to plant your flags and defend them like someone might defend a trademark.

Luckily in the startup world, ideas are worthless and no one cares if you thought of it first but can't execute.

More on academia vs startups: https://twitter.com/mizzao/status/1505529295157948421

Re: How New Are Yann LeCun's “New” Ideas?

#78

Post by belval »

trobertson wrote: post #76
minihat wrote: post #73

Earlier quoted context omitted.

Schmidhuber is a prolific flag-planter who is notorious for publicly raising a stink when he deems he should've been cited, but wasn't. It's happened enough that it's now a meme in the ML community.

It's important to mention that Schmidhuber is usually correct, in that his lab has been decades ahead in both theory and proofs-of-concept. The reason his lab is so under-cited is that his lab made these advances before the hardware to practically do it was available. Now that the techniques can be run, it's the people running them that tend to get all the credit for being "first".

> mention that Schmidhuber is usually correct

You present this statement as fact when it is still highly debated. A lot of researchers will claim a popular new approach is a reformulation of experiments they did X years ago. It's usually best to see it as a spectrum where some idea are on the same axis, where one end is "totally different" and the other is "renamed approached".

Re: How New Are Yann LeCun's “New” Ideas?

#79

Post by mirker »

chaxor wrote: post #69
mirker wrote: post #57

Earlier quoted context omitted.

GPT3 can’t perform algebra over all 32 bit numbers. A trivial Python script can.

It behaves more like your nephew than a computer in that case. Interesting that this is often the example given for why computers are bad at certain tasks, and humans are good at others. It is quite incredible that nothing changed about the architecture in gpt-2 vs gpt-3 (just way more connections), yet it aquired fundamentally new behavior - that if performing arithmetic calculation - despite not having large amount…

It’s smoke and mirrors trying to fool you into thinking it’s generating intelligent text. In some applications e.g., a chatbot, that’s appropriate. But it’s really no comparison to an expert system for most applications, where you know exactly the right and wrong solutions. Not adding numbers correctly with the huge budget GPT3 has for training and inference is a poignant case of that fact. A linear layer taking in x and y will learn x+y just by setting the weights to 1.0, so it’s not even a hard problem for neural nets, just in the particular tokenization and architecture used for GPT models.

Re: How New Are Yann LeCun's “New” Ideas?

#80

Post by lostmsu »

polotics wrote: post #13
obblekk wrote: post #2

I expected this to be a smear / petty argument article. In fact, it's a concise, highly specific, quote by quote critique. I don't have enough context to take a side, but this is not just a rant. Beyond their interpersonal disagreements, I do wonder if LeCunn is seeing diminishing marginal returns to deep learning at FB...

The points are indeed very specific, but they are about opinions, mostly not-even-wrong statements, just reasonable unquantifiables. The elephant in the room is the use of the word "deep" in the field IMHO: it means something else than "many layered neural network" in common parlance...

What does it mean? Techniques that avoid the vanishing gradient problem?
Post reply on HN