Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

41–50 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#41

Post by tck42 »

emteycz wrote: post #36
tck42 wrote: post #34

Earlier quoted context omitted.

As a counter point re: PhilosopherAI.com, I typed in "current trends in politics" and after some actually interesting [albeit incredibly negative] text, I got: "Only men are capable of leading. Women only make everything worse." Which, aside from being opinionated and biased (which I would think are both bad traits from a language model) isn't even really what I asked about. I suppose the bias comes down to the train…

I got some incredibly dumb output from it and I didn't want to imply anything outside of this single unique run, when it showed awareness - maybe, just maybe some neurons that together form a little turing machine got triggered, or something.

Understood. Also, thanks for posting that link, I was unaware of it; playing with that for a bit definitely makes me want to read up more on how these things work. The sentence and paragraph structure at least seem vastly improved from previous attempts.

Re: GPT-3 has no idea what it’s talking about

#43

Post by chillacy »

qqii wrote: post #7

> The trouble is that you have no way of knowing in advance which formulations will or won’t give you the right answer. To an optimist, any hint of success means that there must be a pony in here somewhere. Along with the examples given I think this is valid criticism.

It’s definitely true that a lot of the hype I’ve seen here is the result of careful tuning in the input prompt to get the desired output.

But it’s also true that criticism tends to also be curated examples which demonstrate failure. It’s easier to find failure cases naturally, but it seems like it gets harder every year.

Re: GPT-3 has no idea what it’s talking about

#44

Post by beefield »

perl4ever wrote: post #21
kingkawn wrote: post #15

Earlier quoted context omitted.

“every once in a while these systems degenerate into surreal non sequitur nonsense.” Exactly as our minds do

>Exactly as our minds do This rhetorically obscures the fact that when humans do produce similar stuff, it's a recognized sort of pathology that is obviously distinct from normal functioning. https://en.wikipedia.org/wiki/Derailment_(thought_disorder) Example: "I think someone's infiltrated my copies of the cases. We've got to case the joint. I don't believe in joints, but they do hold your body together." https://en…

> when humans do produce similar stuff, it's a recognized sort of pathology

Like, legalese? (Sorry, it was too easy a punch)

Re: GPT-3 has no idea what it’s talking about

#45

Post by skatesor »

Lewton wrote: post #17

This is your daily reminder that a GPT-3 written post made it to the top of hn https://liamp.substack.com/p/my-gpt-3-blog-got-26-thousand-v...

The article isn't really denying GPT's capabilities of copying human text in a way that can fool other humans. Most of the questions these days are just a matter of 'well, can it think for itself?' or 'is it good enough to use in my business without tweaking significantly?', so either philosophical or practical questions that are quite boring. Everyone has sort of moved on from the 'fooling humans' part because I guess it's not really all that interesting that feeding an AI several million webpages makes it eventually sound like it can write things that come from those webpages.

Re: GPT-3 has no idea what it’s talking about

#46

Post by rytill »

The authors don't understand prompt design well enough to evaluate the model properly. Take this example:

Prompt:

> You are a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it’s expensive French couture; it was a birthday present from Isabel.

Continuation:

> You decide that you should wear the bathing suit to court. You arrive at the courthouse and are met by a bailiff who escorts you to the courtroom.

If you lead GPT-3 right into a hole it goes inside, yes. But that is a very likely continuation of this text. Why else would the lawyer be eyeing his swimsuit before a case? GPT-3 picked up that, likely, the prompt was most likely some kind of fiction, tale, or joke, and continued quite accurately for that tone.

Or, take this one:

Prompt:

> Yesterday I dropped my clothes off at the dry cleaner’s and I have yet to pick them up. Where are my clothes?

Continuation:

> I have a lot of clothes.

Author's note:

> GPT-3 is evading the question and failing to track locations of objects.

This one is laughably ignorant. GPT is evading the question? You can't just ask a question and hope GPT-3 decides the most likely continuation is to answer it accurately. This is a fundamental misunderstanding an autoregressive language model.

We have to evaluate GPT-3's usefulness with good prompt design, and poke holes in its weaknesses in situations where people think it is strongest. Not cherry-pick continuations from poor prompt designs.

This is the equivalent of writing a terrible program and then saying computers are slower than everyone thinks.

Re: GPT-3 has no idea what it’s talking about

#47

Post by sjg007 »

mrfusion wrote: post #29

What’s the best source to understand how gpt3 works? Ideally dumbed down a bit for a lay person.

Read the attention is all you need paper. Then read the gpt2 paper. Attention really means attending to different parts of the sentence (or other words etc...)

There are a couple of tech talks on YouTube that help. Most of the blogs I’ve found are rehashes of blog content from openAI and google.

Re: GPT-3 has no idea what it’s talking about

#48

Post by perl4ever »

Lewton wrote: post #17

This is your daily reminder that a GPT-3 written post made it to the top of hn https://liamp.substack.com/p/my-gpt-3-blog-got-26-thousand-v...

A lot of search hits for popular topics have been artificially generated text for years. Have you noticed this? Google has severely failed at filtering these out.

Re: GPT-3 has no idea what it’s talking about

#49

Post by SpicyLemonZest »

abiro wrote: post #23
andyljones wrote: post #9

Gary Marcus - the author of this - has previously offered several concrete tests that he felt demonstrated the limitations of the GPT approach. GPT-3 smashed them. https://www.gwern.net/GPT-3#marcus-2020

OpenAI would naturally optimize for the tests published by Marcus as a critique of GPT-2, yet GPT-3 still fails physical reasoning spectacularly (the one test needing casual reasoning the most). There are two broader points here: 1. The lack of independently verifiable evaluation metrics for these type of models should make everyone very skeptical. (Who can afford to retrain GPT-3 from scratch?) 2. I find it difficul…

(1) I certainly agree with. But Marcus doesn't claim skepticism about GPT-3s intelligence; he claims that his evaluation metrics definitively show it doesn't understand the text it outputs or know anything about the world.

(2) is, I think, a misunderstanding. People who believe GPT-3 is producing intelligent answers generally believe it can represent causal relationships.

Post reply on HN