OpenAI's GPT-3 may be the biggest thing since Bitcoin • Hacker News Board

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#331

Post by thu2111 » Sun, Jul 19, 2020, 7:59 AM UTC

threeseed wrote: post #61

rvz wrote: post #2

> I further predict that this will spark a creative gold rush among talented amateurs to train similar models and adapt them to a variety of purposes, including: mock news, “researched journalism”, advertising, politics, and propaganda. The first mention of 'Elon Musk' (who left the board) and this sentence alone gave me the tip-off that GPT-3 had generated that (and the whole blog) and it's following prediction make…

> no-one can train GPT-3 in any acceptable time except for those with access to large GPU/ASIC compute power That's not true. a) Premium AWS customers, for example, can request to have instance limits removed which then gives you access to all of the AWS GPU-enabled instances available worldwide. b) People ramble on about GPUs but Intel DLBoost/AVX-512 enabled CPUs can get you comparable performance to a medium end G…

Do you have any links I can read in the latter claim? Using avx512 for ai with comparable power to a GPU?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#332

Post by simonkafan » Sun, Jul 19, 2020, 8:01 AM UTC

I agree that the text generation has improved dramatically. But there is a big gap between "generating gramatically correct sentences" and "sentences that express an elaborated idea". For example:

> I was thinking of how cool it would be to build a Twitter-like service where the only posts are GPT-3 outputs.

Why would I need that? There are better ways for entertainment.

> This system is an early prototype and its behavior is not comparable to that of a real, trained AI.

Nonsense. This made it clear for me that the author (= GPT-3) has no real clue what it's talking about.

Even though this looks like we got one step closer to text understanding - we did not. We just got better in obfuscating that those algorithms have no real sense and clue of what they talk about.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#333

Post by pyb » Sun, Jul 19, 2020, 8:02 AM UTC

gomoboo wrote: post #111

Alex3917 wrote: post #63

Earlier quoted context omitted.

> Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs. I guessed it was fake before getting to the end, not from the content, but from the fact that all the sentences are roughly the same length and follow the same basic grammatical patterns. Real people purposely mix up their sentence structure in order to keep…

I did not guess that it was fake but skipped to the bottom because the article did not seem to be worth reading. It felt like the author was not moving anywhere with their words. I laughed out loud at the reveal that it was written programmatically.

You raise a good point. The Internet has trained us to skim any text that seems pointless or just unsufficiently insightful. So it turns out we have already built up some "mental defences" against GPT-3.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#334

Post by Scarblac » Sun, Jul 19, 2020, 8:02 AM UTC

Alex3917 wrote: post #136

roenxi wrote: post #121

Earlier quoted context omitted.

> Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. I'm going to call this goalpost shifting. This article is better writing than some % of humans, theory of mind or otherwise. The AI has comfortably surpassed Timecube-level writing and is entering the pool of passes-for-a-human. 'Sounds human' is a spectrum that starts with the ment…

> I'm going to call this goalpost shifting. That's completely fair. On the other hand, without a theory of mind it can't really educate or inspire people, the only thing it can do is maybe trick them about the authorship of something. But once people learn the techniques for identifying this kind of writing, it can't even do that anymore. To me this is like the front end of something, but it still needs a back end. D…

It made me consider: - The existence of this model I hadn't heard of - Bitcoin (sigh) - Testing it out on a forum, trying to become a well known poster - picking a forum with many different types of posters - some of which you dislike

And that got me thinking about what I could do with this thing, whether I should, what I wanted to try out...

So the BS random ideas were still inspiring a bit.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#335

Post by woko » Sun, Jul 19, 2020, 8:05 AM UTC

randomsearch wrote: post #315

thewarrior wrote: post #133

Earlier quoted context omitted.

I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by havi…

That’s interesting, as within a sentence I had dismissed your comment as rambling and moved on to the next one, without thinking it had been generated... but maybe you’re double bluffing me.

Same, except that I skipped straight to the last line to check whether it was a generated text after I noticed the first sentence made no sense (GPT-2 already generates grammatically correct English sentences).

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#336

Post by woko » Sun, Jul 19, 2020, 8:08 AM UTC

yyyk wrote: post #112

meredydd wrote: post #28

I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…

I'm not a very good test case. I briefly skimmed (not expecting very much from a Bitcoin-themed article), read the end, and only then read more carefully. So my first read was brief and biased, and my second was very biased. That said, the content of the computer-generated parts doesn't make much sense even for a Bitcoin-influenced article (what would be the point of paraphrasing your previous post in a forum on a re…

> I briefly skimmed (not expecting very much from a Bitcoin-themed article), read the end, and only then read more carefully.

It sounds to me like you must be an academic, or someone with good habits for being efficient at reading articles.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#337

Post by dorkwood » Sun, Jul 19, 2020, 8:09 AM UTC

It's funny, I was making a running tally in my head of the things I was confused about in the article, or the threads the author started that they never wrapped up. For example, I was ready to come here and challenge the author on this point:

> I could not stop thinking about the applications of such a technology and how it could improve our lives.

I didn't actually think that I was reading generated content, though. I guess there's enough confusing writing online that this article wasn't too much of an outlier.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#338

Post by p1esk » Sun, Jul 19, 2020, 8:11 AM UTC

Scarblac wrote: post #320

O_H_E wrote: post #272

Earlier quoted context omitted.

Ok, now this is getting deep and I don't like it.

I assume both previous comments, and this one, are also GPT-3? Edit: it is amusing to think that soon the way to distinguish them will be that human comments have weird errors caused by smartphone keyboard "spell checking" in them...

You really think my comment above was GPT-3 generated? Wow. Did I really make so little sense?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#339

Post by lowdose » Sun, Jul 19, 2020, 8:14 AM UTC

AnimalMuppet wrote: post #225

CamperBob2 wrote: post #166

Earlier quoted context omitted.

On the other hand, without a theory of mind it can't really educate or inspire people I wouldn't agree with that, either. How often have we heard of someone gaining useful insights by considering ideas that were misapplied or just plain wrong? Entire branches of physics have evolved that way. As far as successful presidential campaigns are concerned... well, let's not even go there. If there's such a thing as a 'theo…

I think I disagree. For example, I delayed in writing this comment because the cat was on my lap, and I couldn't fit the laptop and the cat both. You get that. I know you do, even if you don't own a cat, and even if you're reading this on a phone or a desktop. GPT-3 does not understand about the cat. To GPT-3, they're just words and phrases that occur in the vicinity of each other with certain probability distributio…

> But you the human understand, because you have a lap, and you've at least seen a cat on a lap.

You really think GPT-3 never came across a comment about a cat in lap? 50% of all the pictures on the internet are cats sitting on people. GPT-3 doesn't need to understand it to echo this common knowledge.

Airplanes don't look like birds at all but they do fly.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#340

Post by ascar » Sun, Jul 19, 2020, 8:15 AM UTC

geofft wrote: post #238

6nf wrote: post #233

Earlier quoted context omitted.

How many attempts did it take or did you just choose the first one? I have to admit, this is passing my turing test...

Really? I got about halfway through and realized that the comment had no point . If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary. Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....

Also both comments made me suspicious half way through and I scrolled to the bottom to check for a GPT-3 note. Without that note I would definitely have regarded it as incoherent rambling by a human.

Especially the second comment can be coherently interpreted with some good will and a cynical view of the humanities and philosophy. The "author" could say that once GPT-3 can write humanities papers it will quickly make humanity scientists redundant and that humanities scientists are philosophers is not important and doesn't warrant a job alone ("they don't actually do anything"). Eventually it shifts that this is the fault of science working too well (GPT-3 being a product of science)

It's not a consistent argument, but without the context of these comments being GPT-3 it would have totally passed my turing test, just not my sanity test.