Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun on GPT-3

facebook.com

Post reply on HN
Print view View original

161–170 of 253 posts

Re: Yann LeCun on GPT-3

#161

Post by ponker »

capableweb wrote: post #27
dmurray wrote: post #13

Earlier quoted context omitted.

I didn't need to log in to read the article at the original URL, though I had to close a cookie-wall and another modal prompting me to enjoy Facebook better by signing in. Edit: in Ireland, on Firefox desktop

And I couldn't read the article without logging in to my Facebook account. Facebook seems to put different restrictions depending on where you live. I'm based in Western Europe and never been able to read anything from Facebook without logging in. Same for Instagram.

I think Facebook makes a guess about whether you have a Facebook account or not (or, whether you are likely to log into it) and throws up the wall accordingly. If you'll just bail, they'll show it to you anyways. If they think they can force you to log in, they will.

Re: Yann LeCun on GPT-3

#162

Post by FatalLogic »

azinman2 wrote: post #109
FatalLogic wrote: post #38

Earlier quoted context omitted.

You're correct. It's only autocomplete on steroids. But I think it's remarkable that something with the very simple goal of autocomplete can, for a few sentences, sound almost alive

Because we as humans easily anthropomorphize. [1] [1] https://en.wikipedia.org/wiki/The_Media_Equation

Yes, though our tendency to anthropomorphize also helps makes other people human

Re: Yann LeCun on GPT-3

#163

Post by PaulHoule »

thomas_rm wrote: post #49
PaulHoule wrote: post #33

Earlier quoted context omitted.

To remind people: Yann LeCun is an engineering superstar who was working on neural networks at Bell labs long before they were cool. Those handwritten digits that are a scourge today (e.g. 'would any of these methods work on a different set of symbols?' is unasked) came from a competition to develop a commercial zip code reader for the U.S. Postal Service post back in the day of the Apple Newton. He won it! I was a b…

To remind people: Yann LeCun worked on artificial neural networks (ANN) during the period where they were actively shunned by most of the scientific community. You could barely publish a paper on ANN. Just to demonstrate, one the most common books during period, "Artificial Intelligence: A Modern Approach, 2nd ed" by Norvig, 1080 pages, has less than one (1!) page dedicated to ANNs. I personally think Norvig is an id…

Marvin Minsky's 1969 book "Perceptrons"

https://en.wikipedia.org/wiki/Perceptrons_(book)

applied rigorous math (e.g. when computer science was new) to prove that a certain kind of single-layer neural network couldn't solve certain problems. (Can't learn XOR) It is like proving that it takes N log N comparisons to sort N items.

This dampened interest in neural networks for a long time but the "geometrical thinking in hyperdimensional space" is what the field is all about today.

Re: Yann LeCun on GPT-3

#164

Post by wongarsu »

landryl wrote: post #137
neural_thing wrote: post #14

Earlier quoted context omitted.

The humans spent their time building a hideously difficult classification model. Out of the box GPT-3 worked better than the result of a year of their work.

That's interesting, GPT-3 can do classification too? Or did I misunderstood and you meant your engineers used classification to build a language model that didn't perform as well as GPT-3 (which is less surprising indeed) ?

GPT-3 can do classification. For example you can give it a prompt like "Hacker News is a website. Excel is a Windows program. Visual Studio is a Windows program. Safari is a Mac program. CPU-Z is", and even GPT2 will complete this with "a Windows program" (with GPT2 you need to try multiple times, discard useless results and average what's most common, but it works and is straight-forward to automate).

Re: Yann LeCun on GPT-3

#165

Post by xiphias2 »

tiborsaas wrote: post #55
st1x7 wrote: post #2

It's nice to hear from someone who knows what they're talking about that GPT-3 is just a fancy and expensive autocomplete. The hype in some circles about it went as far as comparing it to AGI at some point which is just ridiculous.

You are just a fancy and efficient autocomplete too. When you speak or write, some words have a higher probability than others. You pick alternatives, but they are limited. Of course there are more layers in the human mind, but GPT-3 is a really impressive milestone towards AGI. It's so easy to downplay every advanced tech, it's actually fun. Planes? Just a flying metal tube. Self landing rockets? Just applied physic…

AI / ML? Just linear algebra... I heared that one so many times

Re: Yann LeCun on GPT-3

#166

Post by syndacks »

andybak wrote: post #92

I've got a friend who tries to talk me down every time i enthuse about GPT-3 or similar. He seems to think I believe it to me more amazing than it is whereas I struggle to convince him that I think I've got a good handle on it's limitations and I still find it mind-bogglingly amazing.

Not sure if this is intentional or not, but this sounds potentially bot-written because of a typo and grammatical error.

Re: Yann LeCun on GPT-3

#167

Post by mkolodny »

0-_-0 wrote: post #93
lavoiems wrote: post #82

Earlier quoted context omitted.

GPT-3 is a generative model, isn't it? Can you explain how you converted GPT-3 to a classification model?

You can add new "heads" to GPT networks and train those heads to use GPT for new applications.

Not with GPT-3. I believe only Microsoft is allowed to do that.

Re: Yann LeCun on GPT-3

#168

Post by disambiguation »

fallous wrote: post #124
disambiguation wrote: post #80

Earlier quoted context omitted.

I don't know if i'd go as far as to agree that "there is no knowledge, only language" .. but I 100% agree one of the key insights from GPT-3 -- why training on language is so effective in the first place -- is that language is tightly coupled to reality

I'm not sure that you can assert that language is tightly coupled to reality, unless you're using the term "reality" to mean something akin to "as one perceives the world" (regardless of whether that perception is correct or not). Most expressions of language that survived from a few thousand years ago are centered around myths, and while those myths may have contained certain moral or ethical lessons (that were and…

a few things:

1. i get where you're coming from

2. yes, language is bottlenecked by human perception, as are all things

3. even the notion of myth and fiction is encoded in language. language is self-descriptive and self-aware and you can separate sense from non-sense.

4. i'm not talking about knowledge or understanding, but of addressing the question of why training on language let's GPT-3 make human-like predictions as if it knows about reality? either it's a fluke, or it's because language as a whole is a model that approximates reality.

Re: Yann LeCun on GPT-3

#169

Post by kerng »

turdnagel wrote: post #142
sillysaurusx wrote: post #105

Earlier quoted context omitted.

And when OpenAI says that your two entirely valid use cases are a safety concern, and denies you api access, what will you do? Better keep those ML engineers handy. If you think this isn’t a concern, I’ve already seen it happen with my own eyes, rather than hearing about it second hand. They encouraged someone to make a writing tool. That someone then spent roughly six weeks prototyping, iterating, and giving constan…

Just so you know, for GPT-3, Microsoft is going to be the exclusive licensee of the API: https://blogs.microsoft.com/blog/2020/09/22/microsoft-teams-...

The code, not api

Re: Yann LeCun on GPT-3

#170

Post by sillysaurusx »

part1of2 wrote: post #156
turdnagel wrote: post #142

Earlier quoted context omitted.

Just so you know, for GPT-3, Microsoft is going to be the exclusive licensee of the API: https://blogs.microsoft.com/blog/2020/09/22/microsoft-teams-...

Um, as an outside observer, what is Open about this OpenAI GPT-3 then if they’re selling exclusive rights?

They were forced to give Microsoft exclusive access, because it was one of the terms of Microsoft's billion-dollar cloud credit investment.

But you can't pay employees with cloud credits, so time will tell whether it was a correct decision. (It probably was. And I exaggerate slightly; the investment included a substantial sum of real dollars too. But most people see that billion dollar investment and think it's all dollars, when in fact it was largely credits.)

Post reply on HN