Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 has no idea what it’s talking about

technologyreview.com

Post reply on HN
Print view View original

311–320 of 323 posts

Re: GPT-3 has no idea what it’s talking about

#311

Post by YeGoblynQueenne »

ma2rten wrote: post #310
YeGoblynQueenne wrote: post #296

Earlier quoted context omitted.

The word was used because it was in the prompt and the prompt was constructed in such a way as to force it to use the new word in the place of an old word. No "understanding" is necessary, other than from the human constructing the prompt who needs to understand how the system works. In any case, it's a language model. It has no ability to "understand" anything. It can compute the probability of a token to follow fro…

I'm using "understood" metaphorically similarly to how people say machine learning models "learn" something when they really just minimize a loss.

Ah, thanks for clarifying. In that case, can you explain the metaphor? What do you mean literally, when you say that GPT-3 "understands" something metaphorically?

Edit: btw, "learning" in the context of machine learning is more of a term of trade with well-established connotations. For example, we have Tom Mitchell's definition of a machine learning system as "a system that improves its performance over time", etc. We don't have similarly established definitions for the "understanding" terminology. Hence my request for clarification. I literally don't understand what you mean that GPT-3 "understands" metaphorically.

Re: GPT-3 has no idea what it’s talking about

#312

Post by Isinlor »

azinman2 wrote: post #177
Isinlor wrote: post #112

Earlier quoted context omitted.

To me you are also just a very good version of a Chinese room. How would you show me that you are not?

You can look up my comment history and find a continuity and a perspective mired in personal experience that a machine cannot (at least currently) replicate.

That proves that you are not GPT-3, but you could still be a Chinese room.

You may be even a biological human and I still won't know if you are a Chinese room.

Maybe since birth you are just sort of sleepwalking? I wouldn't be surprised if there were fully functional people without qualia.

Re: GPT-3 has no idea what it’s talking about

#313

Post by ma2rten »

YeGoblynQueenne wrote: post #311
ma2rten wrote: post #310

Earlier quoted context omitted.

I'm using "understood" metaphorically similarly to how people say machine learning models "learn" something when they really just minimize a loss.

Ah, thanks for clarifying. In that case, can you explain the metaphor? What do you mean literally, when you say that GPT-3 "understands" something metaphorically? Edit: btw, "learning" in the context of machine learning is more of a term of trade with well-established connotations. For example, we have Tom Mitchell's definition of a machine learning system as "a system that improves its performance over time", etc. W…

I'm using "understood" as "can apply in an unseen context". For example if you a human student understood a math concept it means they were able to use it in a new context and have not just memorized how to solve a specific problem.

I think the more interesting question is the definition of meaning. I am thinking about meaning here as the relationship between symbols. So if you can explain what a words means, you can give a definition in terms of other words. If you "understood" what a word means, you have not just memorized the definition but can apply the word in unseen contexts.

Re: GPT-3 has no idea what it’s talking about

#314

Post by azinman2 »

Isinlor wrote: post #312
azinman2 wrote: post #177

Earlier quoted context omitted.

You can look up my comment history and find a continuity and a perspective mired in personal experience that a machine cannot (at least currently) replicate.

That proves that you are not GPT-3, but you could still be a Chinese room. You may be even a biological human and I still won't know if you are a Chinese room. Maybe since birth you are just sort of sleepwalking? I wouldn't be surprised if there were fully functional people without qualia.

If I am, I’m the best Chinese room created yet.

Re: GPT-3 has no idea what it’s talking about

#315

Post by microtherion »

Rioghasarig wrote: post #280
amelius wrote: post #238

Earlier quoted context omitted.

> For OpenAI to become a healthy and profitable business, GPT-3 will require them to generate ~50-300 million dollars from the model. On top of that, does anyone have an idea for what practical applications the model could be used? So far I've only seen the model being used to confuse people; how would one turn that into an ethical business? It seems to me that the "BS route" is indeed the logical course.

I actually just sent out an API request for a particular application that I believe GPT-3 should be capable. I'm a university mathematics instructor. I find that a lot of students struggle with doing proofs and I think GPT-3 can help with that. A proof is essentially a sequence of logical inferences. I believe that, when given a proof written in natural language GPT-3 should be capable of detecting the logical implic…

That's probably precisely an area you would NOT want to work with GPT-3. As Julian Togelius put it:

> GPT-3 often performs like a clever student who hasn't done their reading trying to bullshit their way through an exam. Some well-known facts, some half-truths, and some straight lies, strung together in what first looks like a smooth narrative.

https://twitter.com/togelius/status/1284131360857358337?s=20

Re: GPT-3 has no idea what it’s talking about

#316

Post by YeGoblynQueenne »

ma2rten wrote: post #313
YeGoblynQueenne wrote: post #311

Earlier quoted context omitted.

Ah, thanks for clarifying. In that case, can you explain the metaphor? What do you mean literally, when you say that GPT-3 "understands" something metaphorically? Edit: btw, "learning" in the context of machine learning is more of a term of trade with well-established connotations. For example, we have Tom Mitchell's definition of a machine learning system as "a system that improves its performance over time", etc. W…

I'm using "understood" as "can apply in an unseen context". For example if you a human student understood a math concept it means they were able to use it in a new context and have not just memorized how to solve a specific problem. I think the more interesting question is the definition of meaning. I am thinking about meaning here as the relationship between symbols. So if you can explain what a words means, you can…

Thanks for clarifying what you meant by "undestanding". I think it's a bit too broad of a criterion, for example in the video above Morris is using "Tnetennba" in a new context but there's no way to tell whether he really knows what it means (the joke is not that he doesn't know, but exactly that using "Tnetennba" in the kind of sentence he uses it does not elucidate the meaning of the word).

>> So if you can explain what a words means, you can give a definition in terms of other words.

Suppose I give you the following mapping between symbols: a -> p, c -> r, d -> k, e -> j.

Now suppose I give you the phrase: "a a a c a d e e a c"

I gave you a definition of each symbol in the phrase in terms of other symbols. What does the phrase mean? Alternatively, what do the symbols, themselves, mean?

Obviously, you can't say. Being able to give the definition of a word in terms of other words presupposes you understand the other words, also. So, just because a language model is using a word doesn't mean it knows its meaning- only that it uses the word.

Re: GPT-3 has no idea what it’s talking about

#317

Post by ma2rten »

YeGoblynQueenne wrote: post #316
ma2rten wrote: post #313

Earlier quoted context omitted.

I'm using "understood" as "can apply in an unseen context". For example if you a human student understood a math concept it means they were able to use it in a new context and have not just memorized how to solve a specific problem. I think the more interesting question is the definition of meaning. I am thinking about meaning here as the relationship between symbols. So if you can explain what a words means, you can…

Thanks for clarifying what you meant by "undestanding". I think it's a bit too broad of a criterion, for example in the video above Morris is using "Tnetennba" in a new context but there's no way to tell whether he really knows what it means (the joke is not that he doesn't know, but exactly that using "Tnetennba" in the kind of sentence he uses it does not elucidate the meaning of the word). >> So if you can explain…

There is a bit a grey area, which is why I was careful say "the model learned/understood/knowns something" in my earlier comments.

However, You seem to be making the Chinese Room argument. If you define meaning such that either no computer program could possibly "understand" meaning or it is unverifiable if it does, I don't think it makes much sense to have a discussion if GPT-3 does. Is there a test that a model could pass that would convince you that it "knows" meaning according to your definition?

Re: GPT-3 has no idea what it’s talking about

#318

Post by voces »

ClumsyPilot wrote: post #276
voces wrote: post #239

Earlier quoted context omitted.

Language models can help the disabled communicate better and faster (fine-tune on their online output, then the BCI can offer better continuations, so the patient does not have to type as much). Fine-tune on educational materials, and a language model could be a 24/7 assistant to students. Fine-tune on psychology or medicine data, and a language model could tell you of all medicine interactions, or act as a better EL…

I did not downvote, but I think it's because you kinda missed the crux of my question is - what is fine-tuning, like what do you have to actually do? Has anyone done it and demonstrated that it can be done on GPT to get useful and reliable output for some domain? If we exclude fake news/ twitter-bots or something equally silly.

> what is fine-tuning, like what do you have to actually do?

For GPT-2 it is as easy as creating a corpus you want to fine-tune on, adding new unseen tokens to the dictionary, and run a finetuning command. https://huggingface.co/transformers/v1.2.0/examples.html#fin...

> Has anyone done it

Many have. AI Dungeon is a great example. Others are experimenting with humor generation, short story synopsis generation/creative writing, and supportive chat bots.

Re: GPT-3 has no idea what it’s talking about

#319

Post by Isinlor »

azinman2 wrote: post #314
Isinlor wrote: post #312

Earlier quoted context omitted.

That proves that you are not GPT-3, but you could still be a Chinese room. You may be even a biological human and I still won't know if you are a Chinese room. Maybe since birth you are just sort of sleepwalking? I wouldn't be surprised if there were fully functional people without qualia.

If I am, I’m the best Chinese room created yet.

I guess you are :) . I will tell you a secret. I suspect I'm a Chinese room too ^^ ...

There are also people with so called total locked-in syndrome i.e. people who can not do any voluntary movements even with their eyes, so there is no way to communicate with them. How would you decide if they are conscious? It may be literally matter of live and death for them. How should we know if they even want to live in their condition?

Re: GPT-3 has no idea what it’s talking about

#320

Post by samatman »

_y5hn wrote: post #304
samatman wrote: post #215

Earlier quoted context omitted.

That's a difficult question to answer but I would have to say "no". When I had my wisdom teeth out, it was under "deep sedation". They use local anaesthetics, and an additional cocktail which produces sedation, but also, prevents the formation of memories. I have one memory that got through, of a molar being ripped out: but for the most part, my sense of that experience is that I wasn't conscious. The thing is, if yo…

It's not that simple: Ask the person under sedation wether they're conscious or unconscious. If they answer, what does that mean?

Sure, and sometimes you can talk to people who are completely asleep.

I think we agree: it's not that simple. To my taste, a total lack of memory formation is on the unconscious side of this fuzzy, permeable boundary.

And that's just humans. When we start talking about complex software which isn't alive in any meaningful sense, all our intuitions should fail us.

They won't; our intuitions are stubborn things. But they are as likely to mislead as to inform.

Post reply on HN