Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

81–90 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#81

Post by garbagetime »

m3kw9 wrote: post #71
3wolf wrote: post #44

It sounds like you have an output that has been generated by a recurrent neural network trained on a large corpus. The more I read about neural networks the more clear it becomes that there are some classes of them that just write some good stuff. The work of Andrej Karpathy on RNNs for poetry and other things comes to mind. Is your GPT-3 actually even a neural network, though? More research is needed! ^written in re…

Weak

Beside the point, really - The interesting fact is that it's a lot better than anything we have previously had.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#82

Post by nefitty »

acdha wrote: post #56
briga wrote: post #45

Earlier quoted context omitted.

But it might be hard to attract talented AI engineers to live in, say, North Korea

What about North Koreans desperate to be part of the privileged elite? They can take courses and learn just like anyone else.

I’m genuinely curious how accessible that sort of material is to the average North Korean.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#83

Post by nikolay »

Bitcoin was a big thing. You mean, in terms of scams and theft? Have you paid via Bitcoin recently? It's neither fast not cheap. For example, sending money with Venmo, Zelle, Cash is instantaneous and free - unlike with Bitcoin, which is pretty expensive. And when you consider the purchase and sale, it's actually very, very expensive and not smooth.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#84

Post by nullc »

I've been using GPT2 as a writing aid for a little while (the 1558M model is a lot better than the smaller ones). Some of my recently highly upvoted HN comments were substantially written by GPT-2. There is quite a bit of art that goes into crafting prompts that produce useful output.

There are a few things about GPT2 that I didn't like (for instance, the number of "quoted text" blocks) but I have found that if I put my own words in the prompts (in particular when they contain long sentences), it makes things work.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#85

Post by oehtXRwMkIs »

quotemstr wrote: post #50
solidasparagus wrote: post #34

Earlier quoted context omitted.

> What are humans except "text generation" machines? The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.

And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.

For most of human history text has not existed. I think you are conflating language with written language which is quite common in the post-Gutenberg age.

I think sentience like most things is a spectrum, so I'm not really sure what you mean by sentient, but I would argue that for most people the bar for sentience is much higher than text prediction. The Chinese room is only one aspect of our minds, and we don't even know what consciousness is.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#86

Post by briga »

garbagetime wrote: post #69
briga wrote: post #45

Earlier quoted context omitted.

But it might be hard to attract talented AI engineers to live in, say, North Korea

Koreans can become talented AI engineers. The DPRK has a history of simply abducting people with the talents that the DPRK requires. A plausible (although unlikely) scenario is that the DPRK abducts a handful of AI experts and forces them to train Koreans in AI. Alternatives are for Koreans to learn abroad or for Koreans to learn online.

I'm not saying that North Koreans cant be good engineers, I'm saying that it's going to be an uphill struggle for NK to compete for tech talent when US/Chinese companies have so much more of the capital required to build these models

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#87

Post by andrewPP »

Oh, and as we've observed before in mohawk, the final page is absolutely delightful. I'm told this particular author's entry was subsequently removed. Hm, let's see what the PR person says. So, other ideas that I want to mention: There's an entry with a huge crop of people pasted in, but other than that, the input is pretty sporadic. Maybe they could be arranged and directed? There's an odd history at the top of the list. Some of these people are that Waylon Jennings. People have been discussed pretty tightly, but that's enough to suggest they're part of that: "Vokoun" and "Carter" seem pretty stable. I can't help but wonder if the "Vokoun" entries could come from the Vokoun journal itself, and be part of the Hall of Mirrors.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#88

Post by dannyw »

Alex3917 wrote: post #63
meredydd wrote: post #28

I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…

> Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs. I guessed it was fake before getting to the end, not from the content, but from the fact that all the sentences are roughly the same length and follow the same basic grammatical patterns. Real people purposely mix up their sentence structure in order to keep…

If that’s the only thing that separates this from human writing; I’m sure it can be influenced easily.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#89

Post by tedeh »

This is horrifying and whenever someone (in this thread and many others) exclaim how this is "cool" and "exciting" I picture a 13 year old boy out with his mate in the woods saying that after firing three 9mm rounds into a tree just after stealing his fathers gun. That is not to disparage these posters, this is quite obviously in a naive sense a "cool" piece of technology but the ramifications in todays already extremely polarized society of seeing this technology end up in the hands of "the wrong people" and used for all kinds of nefarious purposes makes me feel quite uneasy. Now we can't trust text, the most trusted medium in human history, and then what?

Someone else in this thread doubted that we would see AI within their lifetime, but now think there is a 50/50 shot it will happen in the next decade due to GPT-3. I wasn't expecting to see the disintegration of society during my lifetime, and while I don't think there is a 50 % chance of THAT happening in the next decade, these days that just feels much more likely than before.

OpenAI should stop whatever they are doing and create a searchable repository of every piece text generated by this model so that we have a quick way of checking automatically provenance at the very least.

Post reply on HN