Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

351–360 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#351

Post by swayson »

Sirened wrote: post #323
swayson wrote: post #314

It is ironic as those who now post GPT-3 generated content, is essentially biasing the next version of GPT's web training data

I wonder how much data it would take to actual cause noticeable (or triggerable) behaviors in the model. Like I've noticed certain models have been trained off of my university's course captures by their very odd and specific vocabulary/capitalization of specific terms but surely if you're scanning the entire web you'd need a lot more to sway it

Yeah think your intuition makes sense. Probably won't bias it all to much and perhaps, they can filter it as they probably log what is going through the API.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#352

Post by Vinnl »

nbardy wrote: post #9
minimaxir wrote: post #4

I published a response today to the sudden hype urging people to temper their expectations for GPT-3 a bit: https://minimaxir.com/2020/07/gpt3-expectations/ GPT-3 is objectively a step forward in the field of AI text-generation, but the current hype on VC Twitter misrepresents the model's current capabilities. GPT-3 isn't magic.

One of the biggest issues is with cherry-picking. Generative ML results benefit greatly from humans sampling the best results. They are capable of producing astonishing results but don’t do this consistently this has a huge impact on any effort to productize. For example I’ve seen quite a few examples of text->design, text->code, with GPT-3 you could build a demo in a day, but the product will probably be useless if…

Hmm, I don't know. If you're the IRA [1], it sounds like it could be more efficient to have your trolls select plausible-looking comments from the auto-generated ones rather than having them write them themselves all the time.

[1] https://en.wikipedia.org/wiki/Internet_Research_Agency

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#353

Post by 488643689 »

Alex3917 wrote: post #63
meredydd wrote: post #28

I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…

> Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs. I guessed it was fake before getting to the end, not from the content, but from the fact that all the sentences are roughly the same length and follow the same basic grammatical patterns. Real people purposely mix up their sentence structure in order to keep…

Aww c'mon. You guessed it was fake, because it's an article about computer generated articles. Who would read that and not question the content in front? You are not analysing everything you read for oddities in writing style.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#354

Post by croddin »

Scarblac wrote: post #320
O_H_E wrote: post #272

Earlier quoted context omitted.

Ok, now this is getting deep and I don't like it.

I assume both previous comments, and this one, are also GPT-3? Edit: it is amusing to think that soon the way to distinguish them will be that human comments have weird errors caused by smartphone keyboard "spell checking" in them...

But then if the models are trained on that dataset they might make the same errors to better approximate humans on the forum.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#355

Post by saagarjha »

ekianjo wrote: post #97
Rebelgecko wrote: post #57

Earlier quoted context omitted.

How much of a choice do they have? I can't imagine many of the top engineers in North Korea are allowed to leave

When most of your population is borderline starving the number of top engineers can be counted on very few hands in NK.

That being said, North Korea has a fairly decent cyberespionage program from what I hear.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#356

Post by [deleted] »

Scarblac wrote: post #320
O_H_E wrote: post #272

Earlier quoted context omitted.

Ok, now this is getting deep and I don't like it.

I assume both previous comments, and this one, are also GPT-3? Edit: it is amusing to think that soon the way to distinguish them will be that human comments have weird errors caused by smartphone keyboard "spell checking" in them...

[deleted]

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#357

Post by swayson »

phreeza wrote: post #324
swayson wrote: post #314

It is ironic as those who now post GPT-3 generated content, is essentially biasing the next version of GPT's web training data

See also the current top story about low background steel.

Wow, first time I come across the concept. Fascinating, and a bit eerie when we start drawing closer relationships between AI and nuclear tech.

Interesting times.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#358

Post by unnouinceput »

I'm an optimist so shoot me. I believe that AR combined with AI enhancement is the future for human kind. We will become cyborgs, but not in the dystopian usually described in movies/books, but the one usually described in Asimov novels (like Johnny Mnemonic though I know is not Asimov's novel, but no title comes to mind). We will be 100% in control of all devices, so there will still be both evil and good in humanity.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#359

Post by Reelin »

luckylion wrote: post #347
TomMarius wrote: post #319

Earlier quoted context omitted.

What are your thoughts on why nobody made the set of rules yet?

If designers wanted to write texts to create visual designs, they'd be using some form of DSL and learn to code, wouldn't they? I believe the hype is that people think they can replace the designer by "just telling the computer" what they want. I don't believe that will work, as they already have trouble telling a human what they want, and a computer won't really know what to do with "I want it to kind of feel like i…

> people have a hard time writing their ideas about designs & features down, because they don't know what they want

From an academic standpoint, writing is part of the thinking process. If you haven't written it down, you haven't fully thought it through. If it feels difficult, that's probably because your understanding isn't as complete as you thought it was.

From a software development standpoint, implementing something is part of the thinking process. Ever notice how the requirements have a tendency to break as soon as you actually try to implement them? If a spot seems difficult it just means you hadn't really figured it out yet.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#360

Post by killerstorm »

p1esk wrote: post #185
cs702 wrote: post #147

Earlier quoted context omitted.

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

If he didn't put in the last line ("plot twist ...") I'm pretty sure no one here on HN would have guessed it. In fact, while reading that comment I started to wonder why no one has tried to use GPT to generate text one character at a time. Or if someone has, what are the advantages and disadvantages over the BPE approach.

Well, you'd need to train it from scratch to operate on character level. And it would have smaller context, thus lower quality. So if you want same quality, you need much bigger context.

Still, would be an interesting experiment. Gwern swears it would improve stuff, so worth trying and comparing, I guess

Post reply on HN