Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun on GPT-3

facebook.com

Post reply on HN
Print view View original

41–50 of 253 posts

Re: Yann LeCun on GPT-3

#41

Post by gfodor »

solveit wrote: post #36
formercoder wrote: post #16

Earlier quoted context omitted.

High altitude planes going to the moon is a beautiful analogy. I think this is what I’ll use to explain to less technical friends why I think we’re still many years from self driving cars.

The question isn't whether high-altitude planes can go to the moon, it's whether human intelligence is closer to the clouds or to the moon. For all the talk about how language models "just" learn correlations, there's a remarkable dearth of evidence that humans do something qualitatively different.

Exactly - the analogy fails if a few assumptions we have about ourselves or what GPT-3 is actually “doing” are wrong. Until we hit some asymptotic limit on training these kinds of language models, I’m withholding judgement on what such a model will be capable of representing if/when that limit arrives.

Re: Yann LeCun on GPT-3

#42

Post by tmalsburg2 »

vladf wrote: post #21
st1x7 wrote: post #2

It's nice to hear from someone who knows what they're talking about that GPT-3 is just a fancy and expensive autocomplete. The hype in some circles about it went as far as comparing it to AGI at some point which is just ridiculous.

I happen to use slightly less fancy and expensive GPT-2 based autocomplete, and it's amazing. https://tabnine.com

Interesting. As a reading researcher, I imagine that this could potentially introduce subtle and difficult to spot bugs when you get a proposed completion that looks about right i.e. close enough to what you imagined. Has this been an issue in your experience?

Re: Yann LeCun on GPT-3

#43

Post by ozorOzora »

Traubenfuchs wrote: post #28
neural_thing wrote: post #14

Earlier quoted context omitted.

The humans spent their time building a hideously difficult classification model. Out of the box GPT-3 worked better than the result of a year of their work.

How did they react to this as humans with human pride? Sounds painful.

As GP's declarations are backed by air, we can speculate they are self-reported statements by people working on non-business-centric applications.

edit: GP giving more downvotes than proofs

Re: Yann LeCun on GPT-3

#44

Post by otabdeveloper4 »

solveit wrote: post #36
formercoder wrote: post #16

Earlier quoted context omitted.

High altitude planes going to the moon is a beautiful analogy. I think this is what I’ll use to explain to less technical friends why I think we’re still many years from self driving cars.

The question isn't whether high-altitude planes can go to the moon, it's whether human intelligence is closer to the clouds or to the moon. For all the talk about how language models "just" learn correlations, there's a remarkable dearth of evidence that humans do something qualitatively different.

> remarkable dearth

I've never seen a language model that could create language models. (Never mind the hardware that runs them.)

You're using a very loaded and narrow sense of 'human' here.

Re: Yann LeCun on GPT-3

#45

Post by sjg007 »

Makes sense. You need a richer world model associated with the text then is embedded in word choice. You also need analogies and metaphorical reasoning as well.

Re: Yann LeCun on GPT-3

#46

Post by fastball »

dbcurtis wrote: post #19
formercoder wrote: post #16

Earlier quoted context omitted.

High altitude planes going to the moon is a beautiful analogy. I think this is what I’ll use to explain to less technical friends why I think we’re still many years from self driving cars.

I have been using that analogy to explain why Tesla is many years away from a self driving car. Several others are building something that is fundamentally different.

Aren't they rolling out a beta of FSD literally as we speak?

Re: Yann LeCun on GPT-3

#47

Post by typon »

tosh wrote: post #26

A swiss army knife isn't as good at cutting cheese as a cheese knife.

Disingenuous comparison, since in this case people were acting like the Swiss army knife will overthrow the human race and usher in the singularity.

Re: Yann LeCun on GPT-3

#48

Post by fastball »

blackbear_ wrote: post #24
neural_thing wrote: post #14

Earlier quoted context omitted.

The humans spent their time building a hideously difficult classification model. Out of the box GPT-3 worked better than the result of a year of their work.

Just because many more humans spent many more years and many more $$$ building GPT-3 for your convenience.

Right, but GPT-3 can be used generally. That's the difference. It scales because you don't need to build an entirely new model for each different use case.

You just change the prelude and use it for something new.

Re: Yann LeCun on GPT-3

#49

Post by thomas_rm »

PaulHoule wrote: post #33
czzr wrote: post #6

For anyone else who doesn’t want to deal with Facebook, here’s the post: Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do. This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare. GPT-3 is a language model, which means that you feed it a text and ask it…

To remind people: Yann LeCun is an engineering superstar who was working on neural networks at Bell labs long before they were cool. Those handwritten digits that are a scourge today (e.g. 'would any of these methods work on a different set of symbols?' is unasked) came from a competition to develop a commercial zip code reader for the U.S. Postal Service post back in the day of the Apple Newton. He won it! I was a b…

To remind people: Yann LeCun worked on artificial neural networks (ANN) during the period where they were actively shunned by most of the scientific community. You could barely publish a paper on ANN.

Just to demonstrate, one the most common books during period, "Artificial Intelligence: A Modern Approach, 2nd ed" by Norvig, 1080 pages, has less than one (1!) page dedicated to ANNs. I personally think Norvig is an idiot with regards to Artificial Intelligence, and his book (used in 1500 schools in 135 countries and regions) singlehandedly slowed down the progress of AI by a few years, until a new generation of students outgrew this archaic book.

Re: Yann LeCun on GPT-3

#50

Post by Sharlin »

tosh wrote: post #26

A swiss army knife isn't as good at cutting cheese as a cheese knife.

It's the other way around. GPT-3 is a specialized tool that people are hyping up as a general reasoning agent, or at least a major step towards one.
Post reply on HN