101–110 of 554 posts
I finally clued in after this... no one in their right mind would think this was 'cool and would improve our lives'.
Earlier quoted context omitted.
> What are humans except "text generation" machines? The ability to grow and choose our own direction: to choose what our goals are, curiosity, self-awareness, desire. To imply that GPT-3 is anything close to strong AI is kind of ridiculous.
And how do we indicate that any of these processes has occurred except through the medium of language? A sufficiently good text predictor is a sentient mind. I don't believe that there's any experience of consciousness distinct from the use of language.
Not sure there's one I can communicate to you, but I'm perfectly capable of forgetting the word for something and still knowing unambiguously yet wordlessly what it is, that's an experience.
Catching a ball? Running? Experiencing emotions from wordless music? Viewing scenery? Engaging with a computer game? How are they not conscious experiences?
For those responding to the title or first half of the article, you are missing the point! Edit: the comments that are reacting to the article's content are from people who didn't read it to the end, wow.
I do sometimes wish that HN provided an optional “proof of work” field for comments to differentiate between those responding to the title and those who actually took the time to read the article. Something like a “verified” check mark or toggle button. I’ve found that discussions are most insightful when all participants have “done their homework” so to speak. So while it’s amusing that half of the folks didn’t read…
A forum could instead queue up comments and only display them after a half hour has passed. People who come to the article later will still be at a disadvantage but at least this would be something. On the other hand, you'd probably get a lot of duplicate remarks... which would be less than interesting.
Earlier quoted context omitted.
I mean, you could write coherently about GPT3 without ever hearing about it before-- right? You could absolutely write coherently about it with a bit of priming text, ... I mean, that is exactly what you're doing right now. :) Unless you work for open-ai, you've never actually seen GPT3... for all you know for sure it could be some amazing mechanical turk thing postprocessing GPT2 output :)
I suppose I could have guessed that GPT-2 is a plausible name for the predecessor of GPT-3, and the other predicted predecessor name (PTB) is not really correct. But I'm still surprised that it can hit a believable tone about its own capabilities. I don't think I could have done that if you had asked me before GPT-2 was published (which is essentially what we're doing if GPT-3's training data predates GPT-2). I think…
Keep in mind, the prompt specified that it was the biggest thing since bitcoin and had disruptive potential. Given that you wouldn't write that it sucks.
It also did underestimate its own performance:
> ... nor do its predicted replies tend to be relevant or even grammatically correct. A prototype that had predicted replies that were convincing in most cases would be much more impressive than the GPT-3 I describe here, although that would probably require many years of training and many iterations of improvements on the model
It essentially adopts a common convention for hype articles: A pile of breathless exaggerated hype and then a brief conclusion that suggests that the approach hardly works at the current time, at least if you're reading between the lines. :)
I think it's more surprising that it didn't stray further into fantasy. I'd be interested in seeing the author's rejected samples.
I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…
Earlier quoted context omitted.
One of the biggest issues is with cherry-picking. Generative ML results benefit greatly from humans sampling the best results. They are capable of producing astonishing results but don’t do this consistently this has a huge impact on any effort to productize. For example I’ve seen quite a few examples of text->design, text->code, with GPT-3 you could build a demo in a day, but the product will probably be useless if…
I saw a demo of a gpt3 designing an app that looked just like Instagram home feed skeleton. While it seem impressive, but until you show me something more obscure, that was nothing to brag about.
I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box. Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attenti…
GPT-3 is a neat party trick. But the things that'll be done with web archives* in the next 20y will make it look like the PDP-8. ~love, a web archivist * GPT-3 is trained on one