Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3: Language Models Are Few-Shot Learners

arxiv.org

Post reply on HN
Print view View original

101–110 of 212 posts

Re: GPT-3: Language Models Are Few-Shot Learners

#101

Post by wiz21c »

wyattpeak wrote: post #78
spats1990 wrote: post #7

Even though this was the GPT-3-generated text that humans most easily identified as machine-written, I still like it a lot: Title: Star’s Tux Promise Draws Megyn Kelly’s Sarcasm Subtitle: Joaquin Phoenix pledged to not change for each awards event Article: A year ago, Joaquin Phoenix made headlines when he appeared on the red carpet at the Golden Globes wearing a tuxedo with a paper bag over his head that read, "I am…

I don't know if it says something about text generation or human text processing, but whenever I read an example of computer generated text, all through I think "I can't tell this is machine generated, it seems completely natural," and the only giveaway is that at the end I have no idea what it said. It's a pretty eerie feeling. It's as though both the AI and my short-term processing only pay attention to a context o…

That's really well put. And it does reflect how that kind of document is produced. So I'm not surprised at all.

Problem arises when you read that document without really trying to understand it. In that case, it might be enough to trigger some thoughts.

It also appears to me that two way communications, that is, social interactions, will be the only way to form truth. That is, AI produced content could erode some more the trust we put in of newspaper, TV news, etc (all forms of one way communication). Not that we've waited AI to distrust those, but well, on e more nail in the coffin :-)

(this text, although rather unclear, was written by a genuine human :-) )

Re: GPT-3: Language Models Are Few-Shot Learners

#102

Post by [deleted] »

sillysaurusx wrote: post #96
runawaybottle wrote: post #9

Earlier quoted context omitted.

Dude, I’m sorry, but the average person will not know the difference between that and a regular buzzfeed article or YouTube comment. We’re not going to need ad blockers in the future, we won’t even need these visual ads on websites anymore. There will be trained bots that can promote any idea/product and pollute comments and articles. It’s over, we lost. Morpheus: What if I told you that, throughout your whole life,…

Hello. Gwern and I trained the GPT-2 1.5B model that powers /r/SubSimulatorGPT2. https://www.reddit.com/r/SubSimulatorGPT2/ I've been basically living and breathing GPT-2 for ... gosh, it's been 6 months or so. The past few months have been a lot of StyleGAN2 and a lot of BigGAN, but before that, it was very "make GPT-2 sing and dance in unexpectedly interesting ways" type work. I don't claim to know a lot. But occas…

[deleted]

Re: GPT-3: Language Models Are Few-Shot Learners

#103

Post by bigyikes »

sillysaurusx wrote: post #96
runawaybottle wrote: post #9

Earlier quoted context omitted.

Dude, I’m sorry, but the average person will not know the difference between that and a regular buzzfeed article or YouTube comment. We’re not going to need ad blockers in the future, we won’t even need these visual ads on websites anymore. There will be trained bots that can promote any idea/product and pollute comments and articles. It’s over, we lost. Morpheus: What if I told you that, throughout your whole life,…

Hello. Gwern and I trained the GPT-2 1.5B model that powers /r/SubSimulatorGPT2. https://www.reddit.com/r/SubSimulatorGPT2/ I've been basically living and breathing GPT-2 for ... gosh, it's been 6 months or so. The past few months have been a lot of StyleGAN2 and a lot of BigGAN, but before that, it was very "make GPT-2 sing and dance in unexpectedly interesting ways" type work. I don't claim to know a lot. But occas…

Do you have a blog (preferably human-generated)? I really enjoy your writing style.

Re: GPT-3: Language Models Are Few-Shot Learners

#104

Post by abra0 »

sillysaurusx wrote: post #96
runawaybottle wrote: post #9

Earlier quoted context omitted.

Dude, I’m sorry, but the average person will not know the difference between that and a regular buzzfeed article or YouTube comment. We’re not going to need ad blockers in the future, we won’t even need these visual ads on websites anymore. There will be trained bots that can promote any idea/product and pollute comments and articles. It’s over, we lost. Morpheus: What if I told you that, throughout your whole life,…

Hello. Gwern and I trained the GPT-2 1.5B model that powers /r/SubSimulatorGPT2. https://www.reddit.com/r/SubSimulatorGPT2/ I've been basically living and breathing GPT-2 for ... gosh, it's been 6 months or so. The past few months have been a lot of StyleGAN2 and a lot of BigGAN, but before that, it was very "make GPT-2 sing and dance in unexpectedly interesting ways" type work. I don't claim to know a lot. But occas…

Quite insightful, thanks for writing this.

Re: GPT-3: Language Models Are Few-Shot Learners

#105

Post by Erlich_Bachman »

sillysaurusx wrote: post #96
runawaybottle wrote: post #9

Earlier quoted context omitted.

Dude, I’m sorry, but the average person will not know the difference between that and a regular buzzfeed article or YouTube comment. We’re not going to need ad blockers in the future, we won’t even need these visual ads on websites anymore. There will be trained bots that can promote any idea/product and pollute comments and articles. It’s over, we lost. Morpheus: What if I told you that, throughout your whole life,…

Hello. Gwern and I trained the GPT-2 1.5B model that powers /r/SubSimulatorGPT2. https://www.reddit.com/r/SubSimulatorGPT2/ I've been basically living and breathing GPT-2 for ... gosh, it's been 6 months or so. The past few months have been a lot of StyleGAN2 and a lot of BigGAN, but before that, it was very "make GPT-2 sing and dance in unexpectedly interesting ways" type work. I don't claim to know a lot. But occas…

You should definitely put that up as a blog post somewhere, it is very valuable information, both for researchers and random enthusiasts alike. The emotional modality of it adds important information too :).

Re: GPT-3: Language Models Are Few-Shot Learners

#106

Post by md5person »

nafey wrote: post #63
spats1990 wrote: post #7

Even though this was the GPT-3-generated text that humans most easily identified as machine-written, I still like it a lot: Title: Star’s Tux Promise Draws Megyn Kelly’s Sarcasm Subtitle: Joaquin Phoenix pledged to not change for each awards event Article: A year ago, Joaquin Phoenix made headlines when he appeared on the red carpet at the Golden Globes wearing a tuxedo with a paper bag over his head that read, "I am…

What is it about ai generated texts that on skimming through it it makes sense, but if you try to slow down and understand it feels absurd and surreal.

I think it's because the generated text will generally follow a reasonable "structure", it's "framed" relatively well and you (as a person) will recognize those patterns quickly. There's plenty of "glue" used throughout the text. Those are all patterns which we pick up very quickly, and we're used to seeing them in "real text".

Exampels from the comment above include using things like: "A year ago", "made headlines", "Now, it's the {event}, and {name} is at it again. But this time, ...", "{name} was not impressed", "You know, I feel like, I feel like you could have ...", "I don't know if... but...".

Those are all very common in those "online celebrity magazine" type texts...

It's only when you actually read into the stuff that's in-between, you'll come to see it's pretty much a load of nonsense. But that takes a bit more time and slower reading.

Re: GPT-3: Language Models Are Few-Shot Learners

#107

Post by Nursie »

wyattpeak wrote: post #78
spats1990 wrote: post #7

Even though this was the GPT-3-generated text that humans most easily identified as machine-written, I still like it a lot: Title: Star’s Tux Promise Draws Megyn Kelly’s Sarcasm Subtitle: Joaquin Phoenix pledged to not change for each awards event Article: A year ago, Joaquin Phoenix made headlines when he appeared on the red carpet at the Golden Globes wearing a tuxedo with a paper bag over his head that read, "I am…

I don't know if it says something about text generation or human text processing, but whenever I read an example of computer generated text, all through I think "I can't tell this is machine generated, it seems completely natural," and the only giveaway is that at the end I have no idea what it said. It's a pretty eerie feeling. It's as though both the AI and my short-term processing only pay attention to a context o…

> the only giveaway is that at the end I have no idea what it said

So it's a lot like corporate executive speak then?

I agree with your point, it does seem very much like valid speech, but somehow the informational content is missing. It's like speech without the actual comminication part.

Re: GPT-3: Language Models Are Few-Shot Learners

#108

Post by v4dok »

sillysaurusx wrote: post #96
runawaybottle wrote: post #9

Earlier quoted context omitted.

Dude, I’m sorry, but the average person will not know the difference between that and a regular buzzfeed article or YouTube comment. We’re not going to need ad blockers in the future, we won’t even need these visual ads on websites anymore. There will be trained bots that can promote any idea/product and pollute comments and articles. It’s over, we lost. Morpheus: What if I told you that, throughout your whole life,…

Hello. Gwern and I trained the GPT-2 1.5B model that powers /r/SubSimulatorGPT2. https://www.reddit.com/r/SubSimulatorGPT2/ I've been basically living and breathing GPT-2 for ... gosh, it's been 6 months or so. The past few months have been a lot of StyleGAN2 and a lot of BigGAN, but before that, it was very "make GPT-2 sing and dance in unexpectedly interesting ways" type work. I don't claim to know a lot. But occas…

So true, people unfamiliar with the inner workings get amazed but unfortunately reality is not the same. That being said, I can find multiple ways of utilize this in a bad way. Sex chatbots for example, if I was in that business I would use something like this, it would be extremely easy to phish people off.

Re: GPT-3: Language Models Are Few-Shot Learners

#109

Post by antman »

sillysaurusx wrote: post #96
runawaybottle wrote: post #9

Earlier quoted context omitted.

Dude, I’m sorry, but the average person will not know the difference between that and a regular buzzfeed article or YouTube comment. We’re not going to need ad blockers in the future, we won’t even need these visual ads on websites anymore. There will be trained bots that can promote any idea/product and pollute comments and articles. It’s over, we lost. Morpheus: What if I told you that, throughout your whole life,…

Hello. Gwern and I trained the GPT-2 1.5B model that powers /r/SubSimulatorGPT2. https://www.reddit.com/r/SubSimulatorGPT2/ I've been basically living and breathing GPT-2 for ... gosh, it's been 6 months or so. The past few months have been a lot of StyleGAN2 and a lot of BigGAN, but before that, it was very "make GPT-2 sing and dance in unexpectedly interesting ways" type work. I don't claim to know a lot. But occas…

Great comment! Was it generated with GPT2 or GPT3? I understood all sentences but as a whole I will need to revisit.

Re: GPT-3: Language Models Are Few-Shot Learners

#110

Post by ragebol »

KhoomeiK wrote: post #73
nafey wrote: post #63

Earlier quoted context omitted.

What is it about ai generated texts that on skimming through it it makes sense, but if you try to slow down and understand it feels absurd and surreal.

Yup, definitely different from how humans learn. A baby's speech would be almost the complete opposite, grammatically incorrect here and there but constructing a coherent line of thought for the most part.

My thoughts exactly. It seems a person incapable of proper grammar (like a baby) has some concept or thought it wants to express, but can't because it doesn't know the words etc.

These language models seem to know the words and the grammar etc, but lack a underlying concept they want to express.

There are systems that derive 'thought-vectors', but I'd be interested going the other way: somehow create such a 'thought-vector' and generate text to express that thought.

I don't know how to construct a 'thought-vector' of any concept though.

Post reply on HN