Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun on GPT-3

facebook.com

Post reply on HN
Print view View original

141–150 of 253 posts

Re: Yann LeCun on GPT-3

#141

Post by sillysaurusx »

fastball wrote: post #48
blackbear_ wrote: post #24

Earlier quoted context omitted.

Just because many more humans spent many more years and many more $$$ building GPT-3 for your convenience.

Right, but GPT-3 can be used generally . That's the difference. It scales because you don't need to build an entirely new model for each different use case. You just change the prelude and use it for something new.

It sounds like a big deal. What a tempting idea. And a colleague was mildly annoyed with me for how unimpressed I seemed.

But you have to understand, the use cases you mention are shallow and limited. The heart of GPT, the fine-tuning, is gone. And it looks like even OpenAI gave up on letting users fine-tune, because it means they essentially do build an entirely new, expensive model for each use case.

I wanted to make an HN Simulator, the way that https://www.reddit.com/r/SubSimulatorGPT2/ works. But that's far beyond the capabilities of metalearning (the idea that you describe).

Re: Yann LeCun on GPT-3

#142

Post by turdnagel »

sillysaurusx wrote: post #105
neural_thing wrote: post #5

I'm sure his group has done some rigorous research that I can't even understand. But in my experience, the few-shot learner attribute of GPT-3 makes it insanely useful. We have already found several use cases for it, one of which replaces 2 ML engineers. Yes, it's not perfect, but it's pretty good at many things, and REALLY easy to use.

And when OpenAI says that your two entirely valid use cases are a safety concern, and denies you api access, what will you do? Better keep those ML engineers handy. If you think this isn’t a concern, I’ve already seen it happen with my own eyes, rather than hearing about it second hand. They encouraged someone to make a writing tool. That someone then spent roughly six weeks prototyping, iterating, and giving constan…

Just so you know, for GPT-3, Microsoft is going to be the exclusive licensee of the API: https://blogs.microsoft.com/blog/2020/09/22/microsoft-teams-...

Re: Yann LeCun on GPT-3

#143

Post by aaaxyz »

whymauri wrote: post #114
fludlight wrote: post #88

Earlier quoted context omitted.

“AI” replacing the jobs of AI engineers. But we were told it was only going to do that to blue collar work!

Because most "AI engineering" has lost its meaning and is actually data analysis.

Did it ever have a single meaning? Every company I've been through had a different definition of what "AI engineering" should be

Re: Yann LeCun on GPT-3

#144

Post by thomas_rm »

lisper wrote: post #133
thomas_rm wrote: post #115

Earlier quoted context omitted.

As opposed to the AI techniques taught in the AIMA book (KR and logic reasoning), which had plateau'ed in the 70s...? Norvig had to be pretty clueless to decide ANNs are such a dead-end, that they don't deserve even a chapter in his book, where all around him there are biological living proofs that neural networks are probably a pretty good bet for AI... (Note: I held the same opinion in the mid 90s when I reviewed h…

> all around him there are biological living proofs that neural networks are probably a pretty good bet for AI... 100 years ago you would have been arguing that all around you are living proofs that ornithopters are probably a pretty good bet for artificial flight. You would have been wrong about that too.

We seem to be re-enacting the Symbolic vs Connectionist AI debate of the 80s, poorly.

All I'm saying is, Norvig should have been more humble and included a chapter or two about ANNs, with all the research accumulated thus far, instead of betting 100% for the symbolic approach. Let the next generation of students learn both approaches and decide for themselves. It's sad that a whole generation of students was taught AI from this archaic book.

Re: Yann LeCun on GPT-3

#145

Post by fastball »

legolas2412 wrote: post #58
fastball wrote: post #46

Earlier quoted context omitted.

Aren't they rolling out a beta of FSD literally as we speak?

And that is why i dislike the term "self-driving". It makes people think "driverless", which "full self-driving" is not.

If it can get you from A to B without any human input, is the driver not a safety technicality at that point?

What term would you prefer to describe a car that can get you from A to B without your input?

Re: Yann LeCun on GPT-3

#148

Post by Jack000 »

turdnagel wrote: post #142
sillysaurusx wrote: post #105

Earlier quoted context omitted.

And when OpenAI says that your two entirely valid use cases are a safety concern, and denies you api access, what will you do? Better keep those ML engineers handy. If you think this isn’t a concern, I’ve already seen it happen with my own eyes, rather than hearing about it second hand. They encouraged someone to make a writing tool. That someone then spent roughly six weeks prototyping, iterating, and giving constan…

Just so you know, for GPT-3, Microsoft is going to be the exclusive licensee of the API: https://blogs.microsoft.com/blog/2020/09/22/microsoft-teams-...

My understanding is that the exclusivity is with regard to the code, the API will still be offered to the public.

Re: Yann LeCun on GPT-3

#149

Post by fastball »

sillysaurusx wrote: post #141
fastball wrote: post #48

Earlier quoted context omitted.

Right, but GPT-3 can be used generally . That's the difference. It scales because you don't need to build an entirely new model for each different use case. You just change the prelude and use it for something new.

It sounds like a big deal. What a tempting idea. And a colleague was mildly annoyed with me for how unimpressed I seemed. But you have to understand, the use cases you mention are shallow and limited. The heart of GPT, the fine-tuning, is gone. And it looks like even OpenAI gave up on letting users fine-tune, because it means they essentially do build an entirely new, expensive model for each use case. I wanted to ma…

I think the onus is on you to prove that the use cases are shallow and limited. I've seen GPT-3 already being used for diverse and interesting ideas that would not have occurred to me personally.

However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.

Is it AGI? Obviously not. But the vast majority of ML applications don't need to be.

Re: Yann LeCun on GPT-3

#150

Post by sailingparrot »

yorak wrote: post #113
sailingparrot wrote: post #98

Earlier quoted context omitted.

> which is clearly an overstatement meant to clear up hype, but is untrue It all depends on your definition of knowledge. Under a certain definition you could say that GPT-3 knows basically nothing. If someone teaches me to repeat perfectly something very smart in a language I don't know, without explaining to me what that thing is, do I have knowledge about this? The same argument can be made about those kind of mod…

Aka. the Chinese room argument. However, I'm not so sure us people are little more than just pattern matching machines. When I start to talk (or write, as I'm doing now), the words kind of just flow out. I can make the argument, that I understand the "real" world, but do I really?

> I'm not so sure us people are little more than just pattern matching machines

Yes that's what leads to multiple definition of what knowledge really is. Yann LeCun believe we are more than just that, hence why he is saying GPT-3 would have no knowledge.

Post reply on HN