OpenAI's GPT-3 may be the biggest thing since Bitcoin

Post by scoot_718 » Sun, Jul 19, 2020, 3:10 AM UTC

jcahill wrote: post #53

meredydd wrote: post #28

I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…

GPT-3 is a neat party trick. But the things that'll be done with web archives* in the next 20y will make it look like the PDP-8. ~love, a web archivist * GPT-3 is trained on one

Post by O_H_E » Sun, Jul 19, 2020, 3:13 AM UTC

blhack wrote: post #104

Does anybody have a link to a howto on how to install and play around with this?

Post by p1esk » Sun, Jul 19, 2020, 3:18 AM UTC

cs702 wrote: post #147

thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

Post by abernard1 » Sun, Jul 19, 2020, 3:19 AM UTC

natch wrote: post #172

abernard1 wrote: post #135

Earlier quoted context omitted.

> I think you are underestimating what an advance these models are over previous NLP models in terms of quality. Yeah I mean, I agree. But in my opinion, it's a case of "doing the wrong thing right" instead of a more useful "doing the right thing wrong." I grant that these automated models are useful for low-value classification/generation tasks at high-frequency scale. I don't think that in any way is related to int…

This kind of facile moving the goalposts is imho a cheap shot and (not imho, fact) is a recurring phenomenon as we make incremental progress toward AI. Progress is often made with steps that would have been astonishing a few years ago. And every time the bar is raised higher. Rightly so, but characterizing this as doing the wrong thing is missing the point of what we, and the system, are learning. Yes it's not intell…

Post by zaroth » Sun, Jul 19, 2020, 3:20 AM UTC

graham_paul wrote: post #7

rvz wrote: post #2

> I further predict that this will spark a creative gold rush among talented amateurs to train similar models and adapt them to a variety of purposes, including: mock news, “researched journalism”, advertising, politics, and propaganda. The first mention of 'Elon Musk' (who left the board) and this sentence alone gave me the tip-off that GPT-3 had generated that (and the whole blog) and it's following prediction make…

> no-one can train GPT-3 in any acceptable time except for those with access to large GPU/ASIC compute power (OpenAI, Microsoft, Google, NVIDIA, etc.) Any state actor has access to large compute power