Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

221–230 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#221

Post by dllthomas »

evrydayhustling wrote: post #218
typon wrote: post #52

Earlier quoted context omitted.

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

Just because something doesn’t display understanding by responding to your expectation doesn’t mean it doesn’t possess understanding . If you ran into my office with these prompts, the response to each would be “what the hell are you doing in my office?” All behavior is contextualized, and GPT-3’s native context is predicting continuous text, not answering questions. It’s a distracting anthropomorphism to even attemp…

Right. If I went to a human speaker, who presumably possesses "actual understanding", and I said "I have some prompts for you. Please try to continue where they left off" and then provide them a question, it wouldn't be strange that they don't answer it. Do we ever answer a question right after we've asked it? Sure. Do we always?

I wonder if you can force "an attempt" at answering a question (to flagrantly anthropomorphize) by following the question with something like "The answer is ..."

Re: GPT-3 has no idea what it’s talking about

#222

Post by _y5hn »

wizzwizz4 wrote: post #19
_y5hn wrote: post #16

Earlier quoted context omitted.

How can something non-conscious be aware?

How do you know it's non-conscious? (Serious question.) Yeah, it's not close to a human, but…

It stated at the end: "I have concluded that I am not conscious."

1. If it's being truthful, it means it is right and not being aware. 2. If it's lying, it means it's untrustworthy.

Anyways, to properly debate consciousness, you first need to define the concept, or any debate is just meaningless ramblings.

Re: GPT-3 has no idea what it’s talking about

#223

Post by ClumsyPilot »

mordymoop wrote: post #146
chubot wrote: post #141

Earlier quoted context omitted.

What are the models useful for? As best I can tell the only thing would be some kind of GMail-like auto-reply which a human consciously edits, and which has little consequence if it's wrong. Are the models useful for customer service? Like reading a manual or knowledgebase and then answering a customer's questions about a product, and troubleshooting problems? Like about your Android phone, which lots of people have…

GPT-3 involves zero fine-tuning or customization to any purpose. It’s not attempting to be a product, but rather a platform people can use to explore the possibilities of products. With fine-tuning it could do all sorts of domain-specific things that it can only passably do now. We suspect this to be true because we’ve seen how other systems behave with and without fine-tuning.

you are sidestepping the question above - what can it be used for? What kind of fine-tuning is required? Because to me it looks like 'fixing catastrophic errors' not fine-tuning - as anywhere where the text can deliver value, it has to be correct.

AWS/Azure are a platform, and they work reliably. What kind of platform 'sometimes works'?

Re: GPT-3 has no idea what it’s talking about

#224

Post by evrydayhustling »

dllthomas wrote: post #221
evrydayhustling wrote: post #218

Earlier quoted context omitted.

Just because something doesn’t display understanding by responding to your expectation doesn’t mean it doesn’t possess understanding . If you ran into my office with these prompts, the response to each would be “what the hell are you doing in my office?” All behavior is contextualized, and GPT-3’s native context is predicting continuous text, not answering questions. It’s a distracting anthropomorphism to even attemp…

Right. If I went to a human speaker, who presumably possesses "actual understanding", and I said "I have some prompts for you. Please try to continue where they left off" and then provide them a question, it wouldn't be strange that they don't answer it. Do we ever answer a question right after we've asked it? Sure. Do we always? I wonder if you can force "an attempt" at answering a question (to flagrantly anthropomo…

Yeah, the suffix you suggest is an example of what you’ll see folks calling “prompt design”. It’s becoming an important theme in NLP literature, partly as a forensics tool to find out what kinds of knowledge models encode, and partly as a path to getting a single expensive training regime to assist a variety of tasks.

Re: GPT-3 has no idea what it’s talking about

#225

Post by skatesor »

Der_Einzige wrote: post #175
skatesor wrote: post #32

Earlier quoted context omitted.

You have to remember the AI cannot produce sentences or even words that someone else didn't already write. I'd totally agree it is 'aware' if it could meaningfully come to conclusions like these without getting them from someone else. You might say "don't all humans learn things from someone else" which is not really true because at some point there had to be a first person who learned something completely independen…

Uhhh, no. If your model is using subword tokenization like fasttext or... BERT with wordpiece (and the GPT-X models do this...), than you can generate entirely new words. Wasn't there a demo about doing exactly this a few days ago? https://www.thisworddoesnotexist.com/

Again, it's all coming from somewhere. Humans can react to stimuli in the natural environment and produce sounds and turn them into words. GPT has to be spoonfed a dataset a human made at some point.

Re: GPT-3 has no idea what it’s talking about

#226

Post by turing_complete »

"At the party, I poured myself a glass of lemonade, but it turned out to be too sour, so I added a little sugar. I didn’t see a spoon handy, so I stirred it with a cigarette. But that turned out to be a bad idea because it kept falling on the floor. That’s when he decided to start the Cremation Association of North America, which has become a major cremation provider with 145 locations."

I mean that's just brilliant comedic writing.

Re: GPT-3 has no idea what it’s talking about

#227

Post by mannykannot »

hackinthebochs wrote: post #64
typon wrote: post #52

Earlier quoted context omitted.

I think you're kind of proving the OPs point. The argument is that GPT3 has no understanding of the world, just superficial understanding of words and their relationships. If it did have a real understanding, prompt construction wouldn't matter as much, but it clearly does because all GPT3 cares about the structure of sentences, not their meanings.

>If it did have a real understanding, prompt construction wouldn't matter as much This is only true if we assume GPT was never trained on satire or intentionally absurd text. But there's no reason to think this. Because it continues a bad prompt in an absurd or comical way does not demonstrate it doesn't "understand" common facts. If you treat GPT as a conversation bot and expect it to call you out when you give it a…

I think the point here is that it hs no understanding of what you are calling a 'bad' prompt. That does not mean that it lacks any understanding whatsoever (personally, I think it has made a small step in that direction), but it is a valid case of it not understanding something.

Furthermore, I do not think we can assume that if it were trained on examples of satire or intentionally absurd text, it would perform better on such prompts - in fact, I would not be surprised if its performance would deteriorate on many prompts, both straightforward and tricky ones, if given such training.

Now I am wondering if you need a theory of mind before you can begin to understand satire...

Re: GPT-3 has no idea what it’s talking about

#228

Post by ClumsyPilot »

hackinthebochs wrote: post #159
ClumsyPilot wrote: post #151

Earlier quoted context omitted.

Because if you have a working concept of time, space, and modes of transport, you are aware that a a person has been driving for 2 hours, you can easily deduce the handful of possible towns they might arrive at. Indeed we have software that does that. The statistical model will die to combinatorial explosion between billions of possible combinations of locations, times, and modes of transport. In various literature i…

But this isn't pointing to a fundamental limitation of statistical models, only a limitation of the text corpus. If you had a billion pages of text written about some town and the text included descriptions of travel distances and locations, the model should eventually develop a good representation of the town and relative locations. But of course without such a seed of spatial information, it will just make up plaus…

The statistical argument has limitations, for instance, when there are more pieces of data to record than there are atoms in the universe. Then it falls firmly into impossible category.

> But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input.

i don't think this is true at all, many animals have dedicated 'hardware' for navigation that can sense magnetic fields, etc. We seem to be born with spatial awareness that is far beyond what GPT will ever be capable of.

Re: GPT-3 has no idea what it’s talking about

#229

Post by chillacy »

canjobear wrote: post #189
chillacy wrote: post #134

Earlier quoted context omitted.

This is the thesis of Bender 2020 https://www.aclweb.org/anthology/2020.acl-main.463.pdf when evaluating GPT-2. They distinguish between meaning and form. We don't _really_ know what the physical manifestations of meaning and form are in the brain... they're just concepts we invented. If anything, GPT-3 is suggesting that either: 1. Tasks which were previously thought to require meaning actually turn out only to requ…

It seems like Bender & Koller's argument can just as easily prove that humans don't have representations of meaning either. Our brains get input from multiple "languages"---conventional natural languages as well as the neural codes that represent our sensory inputs. All we have access to is the form of these inputs, and so we can never learn the underlying "meaning".

Hmm I think I see what you're saying. Reality is experienced through electrical signals, arguably we're just learning those inputs. But then I (as always I can only speak for myself, everyone else could be a p-zombie for all I know!) also have a qualitative experience of trees, and words, and an experience of meaning and understanding.

If you look too deeply it quickly gets philosophical.

https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

Post reply on HN