Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun on GPT-3

facebook.com

Post reply on HN
Print view View original

101–110 of 253 posts

Re: Yann LeCun on GPT-3

#101

Post by FartyMcFarter »

It would seem that this can be easily analysed scientifically.

To give a simple example: if, hypothetically, someone thought that GPT-3 is good at basic arithmetic (1 plus 1, 1000 times 3 etc.), they can provide a template for how to ask GPT-3 questions about arithmetic. Anyone can then verify that this template results in accurate answers, by asking randomly sampled questions using that template.

This verification method could be applied to pretty much any problem. Has anyone done anything like that?

Re: Yann LeCun on GPT-3

#102

Post by aaron-santos »

cblconfederate wrote: post #72

GPT3 is definitely overrated at this time. Considering how it was built it should not be considered more intelligent than central pattern generators[ https://en.wikipedia.org/wiki/Central_pattern_generator ]. It's just a pattern generator that generates language instead of a walking pattern. Ascribing to this intelligence has led to some comical claims and studies. Let's start building somethign smart on top of this…

Not that I intend to do the research, but I'd love to see a combination of deep frame semantic extraction laid on top of GPT-n. The formal logic constructions associated with frame semantics have a shot at pushing text models at least away from logical and ontological contradictions.

Re: Yann LeCun on GPT-3

#103

Post by computerphage »

FartyMcFarter wrote: post #101

It would seem that this can be easily analysed scientifically. To give a simple example: if, hypothetically, someone thought that GPT-3 is good at basic arithmetic (1 plus 1, 1000 times 3 etc.), they can provide a template for how to ask GPT-3 questions about arithmetic. Anyone can then verify that this template results in accurate answers, by asking randomly sampled questions using that template. This verification m…

The original paper is full of comparisons and benchmarks. It includes a section on arithmetic.

Re: Yann LeCun on GPT-3

#104

Post by surething123 »

tmalsburg2 wrote: post #42
vladf wrote: post #21

Earlier quoted context omitted.

I happen to use slightly less fancy and expensive GPT-2 based autocomplete, and it's amazing. https://tabnine.com

Interesting. As a reading researcher, I imagine that this could potentially introduce subtle and difficult to spot bugs when you get a proposed completion that looks about right i.e. close enough to what you imagined. Has this been an issue in your experience?

Not the person to whom you asked this question, but I'm also a user of TabNine. In my experience, the "recommendations" / autocompletions provided by the tool are usually very short (probably less than 20 characters on average), and I don't use it for terribly complex chunks of code. Where I like it most is in the initialization of common code chunks like `if` and `for` loops, using variables instantiated in nearby preceding lines. It figures out things like `for(customer in customers)` as I'm writing `for`.

Re: Yann LeCun on GPT-3

#105

Post by sillysaurusx »

neural_thing wrote: post #5

I'm sure his group has done some rigorous research that I can't even understand. But in my experience, the few-shot learner attribute of GPT-3 makes it insanely useful. We have already found several use cases for it, one of which replaces 2 ML engineers. Yes, it's not perfect, but it's pretty good at many things, and REALLY easy to use.

And when OpenAI says that your two entirely valid use cases are a safety concern, and denies you api access, what will you do? Better keep those ML engineers handy.

If you think this isn’t a concern, I’ve already seen it happen with my own eyes, rather than hearing about it second hand. They encouraged someone to make a writing tool. That someone then spent roughly six weeks prototyping, iterating, and giving constant feedback. All signals from OpenAI were “Yes, awesome!”

Then one day they simply declined to let them ship. Anything. Anything even resembling “a tool to generate huge quantities of outputs.” Which was, you know, the whole point.

You play, you pay. And I hope you’re ready to pay, because you won’t have your magical genie unless the magical genie’s caretakers believe you are sufficiently worthy.

I cloned as much of OpenAI’s API as I could: https://twitter.com/theshawwn/status/1312299759592333318?s=2...

All that’s left is to reproduce a substantially similar model. Which is doable, but will take time. In the meantime, tread carefully.

Re: Yann LeCun on GPT-3

#106

Post by FartyMcFarter »

computerphage wrote: post #103
FartyMcFarter wrote: post #101

It would seem that this can be easily analysed scientifically. To give a simple example: if, hypothetically, someone thought that GPT-3 is good at basic arithmetic (1 plus 1, 1000 times 3 etc.), they can provide a template for how to ask GPT-3 questions about arithmetic. Anyone can then verify that this template results in accurate answers, by asking randomly sampled questions using that template. This verification m…

The original paper is full of comparisons and benchmarks. It includes a section on arithmetic.

Thanks! The corresponding graphs in the paper show that it's OK at two-digit operations (except multiplication), but it doesn't generalize to bigger numbers. This would seem to support LeCun's statement that there's a lot of over-hyping going on.

Re: Yann LeCun on GPT-3

#107

Post by msamwald »

The original Nabla article is missing information on how they primed GPT-3 for each use-case, and how much effort they put into finding good ways of priming.

All fancy GPT-3 demos seem to rely on good priming.

The time scheduling problems are probably hard limit of GPT-3 capabilities. The "kill yourself" advice, on the other hand, might have been avoided by better priming.

Re: Yann LeCun on GPT-3

#108

Post by andrewprock »

capableweb wrote: post #27
dmurray wrote: post #13

Earlier quoted context omitted.

I didn't need to log in to read the article at the original URL, though I had to close a cookie-wall and another modal prompting me to enjoy Facebook better by signing in. Edit: in Ireland, on Firefox desktop

And I couldn't read the article without logging in to my Facebook account. Facebook seems to put different restrictions depending on where you live. I'm based in Western Europe and never been able to read anything from Facebook without logging in. Same for Instagram.

I had no problem in CA using Firefox.

Re: Yann LeCun on GPT-3

#109

Post by azinman2 »

FatalLogic wrote: post #38
st1x7 wrote: post #2

It's nice to hear from someone who knows what they're talking about that GPT-3 is just a fancy and expensive autocomplete. The hype in some circles about it went as far as comparing it to AGI at some point which is just ridiculous.

You're correct. It's only autocomplete on steroids. But I think it's remarkable that something with the very simple goal of autocomplete can, for a few sentences, sound almost alive

Because we as humans easily anthropomorphize. [1]

[1] https://en.wikipedia.org/wiki/The_Media_Equation

Re: Yann LeCun on GPT-3

#110

Post by sailingparrot »

DavidSJ wrote: post #95
sailingparrot wrote: post #87

Earlier quoted context omitted.

Yes except they were saying the iPhone replaced Nokia's engineers. GPT-3 is not doing what the ML engineers were doing (building models), GPT-3 is the end goal. The company just decided to outsource the work to OpenAI and pay a monthly fee to them instead of salaries to their ML engineers. "We have already found several use cases for it, one of which replaces 2 ML engineers." -> Clearly makes it sounds like GPT-3 can…

From a business perspective, this is an irrelevant distinction. The requirement was satisfied in a different way, i.e. the engineers satisfying the requirement were replaced by GPT-3, the tool which satisfies the requirement. I think everyone understood that.

The thread is not about business perspective, it's about the hype around what GPT-3 is and is not able to do.

One thing GPT-3 is not able to do for example, is replacing 2 ML engineers to build a GPT-3 like model. But OpenAI can do that.

Post reply on HN