Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's GPT-3 may be the biggest thing since Bitcoin

maraoz.com

Post reply on HN
Print view View original

31–40 of 554 posts

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#31

Post by m3kw9 »

nbardy wrote: post #9
minimaxir wrote: post #4

I published a response today to the sudden hype urging people to temper their expectations for GPT-3 a bit: https://minimaxir.com/2020/07/gpt3-expectations/ GPT-3 is objectively a step forward in the field of AI text-generation, but the current hype on VC Twitter misrepresents the model's current capabilities. GPT-3 isn't magic.

One of the biggest issues is with cherry-picking. Generative ML results benefit greatly from humans sampling the best results. They are capable of producing astonishing results but don’t do this consistently this has a huge impact on any effort to productize. For example I’ve seen quite a few examples of text->design, text->code, with GPT-3 you could build a demo in a day, but the product will probably be useless if…

I saw a demo of a gpt3 designing an app that looked just like Instagram home feed skeleton. While it seem impressive, but until you show me something more obscure, that was nothing to brag about.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#34

Post by solidasparagus »

quotemstr wrote: post #25
minimaxir wrote: post #4

I published a response today to the sudden hype urging people to temper their expectations for GPT-3 a bit: https://minimaxir.com/2020/07/gpt3-expectations/ GPT-3 is objectively a step forward in the field of AI text-generation, but the current hype on VC Twitter misrepresents the model's current capabilities. GPT-3 isn't magic.

> AI text-generation "Text generation" undersells it a little bit. What are humans except "text generation" machines? Language is the stuff of reason. GPT-3 has demonstrated capabilities that we believed were exclusive to humanity --- humor, logic, sarcasm, cultural references --- in an automatic and generalizable way. It's far more than a "text generation" system. I look forward to seeing what GPT-4 and GPT-5 can do…

> What are humans except "text generation" machines?

The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#36

Post by qayxc »

thom wrote: post #13
drcode wrote: post #6

Personally, in 2014 I had put the odds of seeing human-level AI during my lifetime at around 10%. Given the latest developments, I would put the odds at fifty-fifty within the next decade.

How would you measure human level AI?

Since intelligence isn't a well-defined term, the default fall-back would be a measurement by proxy.

Humans are usually tasked with solving an IQ-test as the most common proxy.

A human level AI could do just the same. If the AI scores above 50 points, it can be considered human level (though literally retarded).

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#37

Post by BickNowstrom »

derefr wrote: post #23
confeit wrote: post #10

Earlier quoted context omitted.

Giving certain people, highly visible on social media, pre-public access to the model, and letting them cherry pick their completions to post without the prompt or amount of tries, is a smart form of propaganda/hype building/PR management, that we have come to expect from "GPT-2 is too dangerous to release" openAI Sometimes I forget that, while this model was created by scientists, and released with a scientific pape…

> Sometimes I forget that, while this model was created by scientists, and released with a scientific paper, it is essentially a for-profit business product, and such cheap tricks deserve harsh criticism. Sure, but this is akin to seeing bad science journalism and tarring the science itself with the same brush. GPT-3 still factually has certain properties, independently of anyone making grandiose assertions about tho…

It is certainly impressive, and I don't want to discard GPT-3. Just critiquing the (smart) release: make a select few feel special by giving them API access, and watch your product dominate the tech - and news cycle for weeks. You'll have VC money in the bank before showing actual worth or business value.

Maybe a bit simplistic, but I view GPT as a Markov chain text generator, operating on word vectors instead of word tokens, and having a larger look-back. It's like a child copying a joke, because she heard adults laughing about it, but she does not understand the punchline. You wouldn't say that child understands or even displays humor, despite substituting "horse" with "donkey" when retelling the joke.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#38

Post by drcode »

k__ wrote: post #33

I read, it's now all about "priming" and not "training" anymore. What does this mean? Where can I learn about priming?

With GPT-2, you had to (for instance) train the AI with thousands of poems for hours to get the AI ready to write its own poem.

With GPT-3, you can just say "Here are three poems by Dr. Seuss about Grumpy Cat" and then it'll (sometimes) write some convincing poems.

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#39

Post by jbhouse »

m3kw9 wrote: post #31
nbardy wrote: post #9

Earlier quoted context omitted.

One of the biggest issues is with cherry-picking. Generative ML results benefit greatly from humans sampling the best results. They are capable of producing astonishing results but don’t do this consistently this has a huge impact on any effort to productize. For example I’ve seen quite a few examples of text->design, text->code, with GPT-3 you could build a demo in a day, but the product will probably be useless if…

I saw a demo of a gpt3 designing an app that looked just like Instagram home feed skeleton. While it seem impressive, but until you show me something more obscure, that was nothing to brag about.

Please please please post a link to that video. It sounds super interesting

Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin

#40

Post by gk1 »

qeternity wrote: post #19

Oh man. I don’t know where to begin. I’ll just say that this analysis is predicated on someone who thinks Bitcoin has proven anything in the real world. We’re way past BTC being a subculture experiment. Everyone knows about it. And still nobody (in the statistical sense) uses it.

You can begin by reading the article to the end. :)
Post reply on HN