Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How New Are Yann LeCun's “New” Ideas?

garymarcus.substack.com

Post reply on HN
Print view View original

51–60 of 90 posts

Re: How New Are Yann LeCun's “New” Ideas?

#51

Post by jonathankoren »

benreesman wrote: post #20

These guys know better than to rev the tachometer up in the lay press talking about AGI and “achieve human level intelligence” and stuff. This fluff, unfortunately, sells and so when you’ve got an ego big enough to be talking this way in the first place I suppose you feel like you have to do it? Machine learning researchers optimize “performance” on “tasks”, and while those terms are still tricky to quantify or even…

My personal thoughts about AGI is that we’ll never “achieve” it for pretty much the same reasons you, and philosophers for hundreds, if not thousands, of years have said. We can’t even be sure that anyone else is conscious and intelligent, or just a clever facsimile.

As AI (in the broadest sense) has developed, we always end up moving the goal posts. Sometimes this is because we genuinely don’t know what is difficult and what is easy due to several billion years of evolution. But some of this is because we know how the system works, and so it can’t be “intelligence”.

I think of it as like a magic trick. When you watch a someone do an illusion well, it’s amazing. They made the coin disappear. It’s real magic! But then you find out all they did was stick in their pocket, or used a piece of elastic, and then “magic” is gone.

Essentially this is partially what the Chinese Room is about. You think the Chinese speaker is real, but then you find out it’s just some schlub executing finite state machine.

Re: How New Are Yann LeCun's “New” Ideas?

#52

Post by andrepd »

bjourne wrote: post #49

So the idea is that statistical language modelling is not enough. You need a model based on logic too for "real" artificial intelligence. I wonder what the evidence for this claim is? Because the inferences and reasoning GPT3 is already capable of is incredible and beats most expert systems that I know of. And GPT4 is around the corner, Stable Diffusion was published like only a few months ago. I don't see why not mo…

> Because the inferences and reasoning GPT3 is already capable of is incredible and beats most expert systems that I know of.

This is patently FALSE. You can, however, re-run a given prompt 10+ times, tweaking and nudging it into the direction you know you want, until it produces a seemingly miraculously deep result (by pure chance).

Rinse and repeat a dozen times and you have enough material for a twitter thread or medium post fawning over gpt-3.

Re: How New Are Yann LeCun's “New” Ideas?

#53

Post by robg »

Old enough to remember when Marcus was picking out-of-scope fights with parallel distributed processing models and scholars. On the one hand, he’s right, symbol manipulation is different in kind, not degree. On the other, we’ve known that since the dawn of neural networks. To claim credit for theoretical gaps that others try to fill in practice seems petty and myopic.

Re: How New Are Yann LeCun's “New” Ideas?

#54

Post by mardifoufs »

andrepd wrote: post #52
bjourne wrote: post #49

So the idea is that statistical language modelling is not enough. You need a model based on logic too for "real" artificial intelligence. I wonder what the evidence for this claim is? Because the inferences and reasoning GPT3 is already capable of is incredible and beats most expert systems that I know of. And GPT4 is around the corner, Stable Diffusion was published like only a few months ago. I don't see why not mo…

> Because the inferences and reasoning GPT3 is already capable of is incredible and beats most expert systems that I know of. This is patently FALSE. You can, however, re-run a given prompt 10+ times, tweaking and nudging it into the direction you know you want, until it produces a seemingly miraculously deep result (by pure chance). Rinse and repeat a dozen times and you have enough material for a twitter thread or…

I don't necessarily doubt you but can you give me an example of an expert system that is more capable ?

Re: How New Are Yann LeCun's “New” Ideas?

#55

Post by oldgradstudent »

adamsmith143 wrote: post #25
oldgradstudent wrote: post #4

Earlier quoted context omitted.

Any specific reason this is relevant to the arguments in his post?

Because Marcus isn't a practitioner and never has been. He's a public intellectual from a different field acting like he's an AI expert. You would never listen to criticisms of a Physics theory from a Biologist and you shouldn't listen to criticisms of Neural Networks from a Psychologist. He's proven time and time again that he doesn't understand the methods at work and doesn't even seem interested in trying to do so…

> You would never listen to criticisms of a Physics theory from a Biologist and you shouldn't listen to criticisms of Neural Networks from a Psychologist.

Why? If their arguments are sound, why shouldn't we listen to them?

Where is this silly credentialism coming from?

Re: How New Are Yann LeCun's “New” Ideas?

#56

Post by oldgradstudent »

frisco wrote: post #5
oldgradstudent wrote: post #4

Earlier quoted context omitted.

Any specific reason this is relevant to the arguments in his post?

What is he even arguing here? That he has been cheated out of some kind of credit? Credit for what? Afaict he has never actually shown something novel based on his ideas to work in a way that has mattered.

His argument may be silly or pathetic or false or even true, but it has nothing to do with how long he held the SOTA record of any known machine learning task.

Re: How New Are Yann LeCun's “New” Ideas?

#57

Post by mirker »

mardifoufs wrote: post #54
andrepd wrote: post #52

Earlier quoted context omitted.

> Because the inferences and reasoning GPT3 is already capable of is incredible and beats most expert systems that I know of. This is patently FALSE. You can, however, re-run a given prompt 10+ times, tweaking and nudging it into the direction you know you want, until it produces a seemingly miraculously deep result (by pure chance). Rinse and repeat a dozen times and you have enough material for a twitter thread or…

I don't necessarily doubt you but can you give me an example of an expert system that is more capable ?

GPT3 can’t perform algebra over all 32 bit numbers. A trivial Python script can.

Re: How New Are Yann LeCun's “New” Ideas?

#59

Post by etaioinshrdlu »

As far as I know our brains are mostly unchanged for thousands of years. So any novel ideas anyone has are a result of standing on the shoulders of giants, idea-wise and technology-wise, so it seems rather silly to give any individual the lion's share of the credit for any new idea of any kind, anywhere.

Re: How New Are Yann LeCun's “New” Ideas?

#60

Post by mirker »

adamsmith143 wrote: post #27
obblekk wrote: post #2

I expected this to be a smear / petty argument article. In fact, it's a concise, highly specific, quote by quote critique. I don't have enough context to take a side, but this is not just a rant. Beyond their interpersonal disagreements, I do wonder if LeCunn is seeing diminishing marginal returns to deep learning at FB...

Diminishing returns? Have you read the Gato, Palm, Stable Diffusion, etc. papers? Progress is racing ahead. Nothing is stalling... the only thing stopping progress from accelerating even faster is data.

Many of these scaling patterns are logarithmic with respect to data size. You can only double the dataset size so many times that it’s really not clear the scaling will continue.
Post reply on HN