OpenAI's GPT-3 may be the biggest thing since Bitcoin

Post by losvedir » Sun, Jul 19, 2020, 12:10 PM UTC

tedeh wrote: post #89

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extre…

Post by polyanos » Sun, Jul 19, 2020, 12:12 PM UTC

killerstorm wrote: post #378

rvz wrote: post #2

> I further predict that this will spark a creative gold rush among talented amateurs to train similar models and adapt them to a variety of purposes, including: mock news, “researched journalism”, advertising, politics, and propaganda. The first mention of 'Elon Musk' (who left the board) and this sentence alone gave me the tip-off that GPT-3 had generated that (and the whole blog) and it's following prediction make…

Why do you believe they won't release the model? It's Open AI after all. Google, NVIDIA and Microsoft also make their models freely available. Google already trained a model which is bigger than GPT-3. There might be some delay, but no fundamental problem with it. Fine-tuning the entire GPT-3 is impossible on a single GPU. But it's still possible to fine-tune specific layers. Plus, I'm sure somebody will release a di…

Post by techdragon » Sun, Jul 19, 2020, 12:18 PM UTC

spion wrote: post #409

setzer22 wrote: post #275

Earlier quoted context omitted.

This was very apparent when reading the generated stories [1]. Especially the shoggoth cat dialogue, I found that one really creepy. The fragment below comes straight out from the uncanny valley: Human: Those memes sound funny. But you didn’t include any puns. So tell me, what is your favorite cat pun? AI: Well, the best pun for me was the one he searched for the third time: “You didn’t eat all my fish, did you?” You…

An instance of a Voight-Kampff test.

Post by pasabagi » Sun, Jul 19, 2020, 12:36 PM UTC

cs702 wrote: post #147

thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…

Post by pgt » Sun, Jul 19, 2020, 12:55 PM UTC

canjobear wrote: post #122

abernard1 wrote: post #96

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner. This does not impress me in the slightest. Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying corr…

I think you are underestimating what an advance these models are over previous NLP models in terms of quality. Before GPT-2 we didn't even have models that could reliably generate grammatical sentences. Now we have things that generate coherent (if not beautiful) paragraphs. It seems easy in retrospect, but some of the smartest people around have been working on this for decades.