Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

91–100 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#91

Post by joe_the_user »

meredydd wrote: post #28

I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…

OK, I read the first sentence and it sounded like a typical poor-quality marketing-blather article so I came here and read your post.

I then reread it and it indeed read like a weird, rambling, incoherent article. Looking at it closely, it had a good many contradictory, meaningless and incoherent sentences.("It is a popular forum with many types of posts and posters.")

The headline, however, seemed about right.

It's true the nonsense in this article is a bit different than the nonsense of a GPT-2 article. But the thing GPT-2 paragraphs sound pretty coherent 'till they suddenly go off the rail. This is more like an article that was never quite on the rails and so it's slightly more internally cohesive. But not "better".

Maybe the article just reflects the author's style. Anyone have a GPT-3 test site link?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#92

Post by pastrami_panda »

quotemstr wrote: post #50
solidasparagus wrote: post #34

Earlier quoted context omitted.

> What are humans except "text generation" machines? The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.

And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.

> I don't believe that there's any experience of consciousness distinct from the use of language.

To me this indicates a very narrow view of consciousness. Consider for a moment the quiet consciousness of the cerebellum for example.

I like the way David F. Wallace put it: 'Both flesh and not'. There's an astounding amount of consciousness that is not bound by language. One can even argue that language might hinder those forms of consciousness from even arising.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#93

Post by yyyk »

threeseed wrote: post #79
yyyk wrote: post #43

Earlier quoted context omitted.

The hype/scare re GPT-2/3 (etc.) is not for their poetry output, but rather for its potential for mass propaganda, telemarketing and so on. We can already get humans to do this stuff, all GPT could give is scale (that's no small deal). However, if the output needs to be curated and edited by humans, the scale and automation is gone - we just get a different manual process, with a modest improvement to speed at cost o…

The truly scary part is SEO where GPT-3 could ruin search engines overnight. Google at this point favours long form content for many search intents. Being able to generate thousands of these pages in one-click is a real problem. Not just because of popular topics e.g. "covid-19 symptoms" but more so for the long tail e.g. "should I drink coffee to cure covid-19".

Quite a lot of SEO already uses simple word generation techniques. It isn't clear GPT-3 is an improvement there - human text recognition might not be whatever Google does.

It may be that Google's algorithms don't care at all how human-like the text is, or that their own recognition algorithm/NN (whatever they use) isn't fooled. Even if it is affected, Google has the money and corpus to build its own competing NN to recognize GPT-3 text.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#94

Post by ekianjo »

coderintherye wrote: post #16
minimaxir wrote: post #4

I published a response today to the sudden hype urging people to temper their expectations for GPT-3 a bit: https://minimaxir.com/2020/07/gpt3-expectations/ GPT-3 is objectively a step forward in the field of AI text-generation, but the current hype on VC Twitter misrepresents the model's current capabilities. GPT-3 isn't magic.

I don't know, this seemed pretty close to magic to me: https://twitter.com/jsngr/status/1284511080715362304 Granted, it seems like there was a lot of behind the scenes work to make that happen.

this is something you could do with NLP already before.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#96

Post by abernard1 »

When I read comments like this--and yes I read the article and understand it was generated by an algorithm--I can't help but think the next AI winter is around the corner.

This does not impress me in the slightest.

Taking billions and billions of input corpora and making some of them _sound like_ something a human would say is not impressive. Even if it's at a high school vocabulary level. It may have underlying correlative structure, but there's nothing interesting about the generated artifacts of these algorithms. If we're looking for a cost-effective way to replace content marketing spam... great! We've succeeded! If not, there's nothing interesting or intelligent in these models.

I'll be impressed the day I can see a program that can 1) only rely on its own limited experiential inputs and not billions of artifacts (from already mature persons), and 2) come up with the funny insights of a 3-year-old.

Little children can say things that sound nonsensical but are intelligent. This sounds intelligent but is nonsensical.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#97

Post by ekianjo »

Rebelgecko wrote: post #57
briga wrote: post #45

Earlier quoted context omitted.

But it might be hard to attract talented AI engineers to live in, say, North Korea

How much of a choice do they have? I can't imagine many of the top engineers in North Korea are allowed to leave

When most of your population is borderline starving the number of top engineers can be counted on very few hands in NK.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#98

Post by garbagetime »

briga wrote: post #86
garbagetime wrote: post #69

Earlier quoted context omitted.

Koreans can become talented AI engineers. The DPRK has a history of simply abducting people with the talents that the DPRK requires. A plausible (although unlikely) scenario is that the DPRK abducts a handful of AI experts and forces them to train Koreans in AI. Alternatives are for Koreans to learn abroad or for Koreans to learn online.

I'm not saying that North Koreans cant be good engineers, I'm saying that it's going to be an uphill struggle for NK to compete for tech talent when US/Chinese companies have so much more of the capital required to build these models

The DPRK doesn't have to compete for tech talent in the same way most countries do, because other states probably won't be able to poach tech talent away from the DPRK.

And how much talent would the DPRK really need to do impactful work with GPT-3 (assuming that it really can be used for be used for nefarious purposes)?

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#99

Post by joe_the_user »

nbardy wrote: post #9
minimaxir wrote: post #4

I published a response today to the sudden hype urging people to temper their expectations for GPT-3 a bit: https://minimaxir.com/2020/07/gpt3-expectations/ GPT-3 is objectively a step forward in the field of AI text-generation, but the current hype on VC Twitter misrepresents the model's current capabilities. GPT-3 isn't magic.

One of the biggest issues is with cherry-picking. Generative ML results benefit greatly from humans sampling the best results. They are capable of producing astonishing results but don’t do this consistently this has a huge impact on any effort to productize. For example I’ve seen quite a few examples of text->design, text->code, with GPT-3 you could build a demo in a day, but the product will probably be useless if…

If OP article was cherry-picked, the tree must not be very productive.

More than cherry-picking, there's the Eliza Effect - it's pretty easy to make people think generated text is intelligent. That text can seem intelligent for a while isn't necessarily impressive at all.

https://en.wikipedia.org/wiki/ELIZA_effect

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#100

Post by nutanc »

The main thing to consider is the author choosing the sentences. The author says they only made a couple( less than 10) edits. But the human input is what is important. I remember something similar when GPT2 came out(I think it was an interview conducted by GPT2, with human tuning).

I have been following the twitter hype on GPT-3. @quasimondo has done some excellent analysis on text generation using GPT-3. For me, when reading GPT-3 text, something is off. It jumps around too much. It does not seem to care. I couldn't read more than 10 sentences if the text is completely GPT-3 generated. But if prompted properly by the human at regular intervals, it sounds ok. Generally it holds a thought upto one paragraph. So maybe the usefulness of GPT-3 is in scenarios like, I have a thought, I want to complete it, I fire up GPT-3, prompt it and generate different results, choose one which is closest to my thoughts and edit it and publish. I am sure that's what this blog author did.

GPT-3 is really good at NLG. But people seem to be extrapolating the results to NLP.

Post reply on HN