Plot twist, all of these HN comments were generated by GPT-3, including this one... dun, dun, dahhh.
OpenAI's GPT-3 may be the biggest thing since Bitcoin
441–450 of 554 posts
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#442Earlier quoted context omitted.
Why do you believe they won't release the model? It's Open AI after all. Google, NVIDIA and Microsoft also make their models freely available. Google already trained a model which is bigger than GPT-3. There might be some delay, but no fundamental problem with it. Fine-tuning the entire GPT-3 is impossible on a single GPU. But it's still possible to fine-tune specific layers. Plus, I'm sure somebody will release a di…
>Why do you believe they won't release the model? It's Open AI after all. Because they want to commercialize the model "to cover the costs of research", it's stated in their blog post FAQ[0]. >Google already trained a model which is bigger than GPT-3. Source?, a fast Google search yielded no results and I'm curious. [0] https://openai.com/blog/openai-api/
They commercialize the API. The model is so large you can't run it on a single GPU, AFAIK. OpenAI developed an infrastructure to run the model efficiently. So many companies would rather use API than deploy the model on their own hardware.
They will probably release the baseline model, not model optimized for deployment. There are many optimizations possible such as precision reduction, pruning, distillation, and they don't have to share these optimizations.
> a fast Google search yielded no results and I'm curious.
List of pre-trained models on huggingface: https://huggingface.co/models
You can see some of them are prefixed with "google". Also ALBERT is from Google Research.
That's just natural language processing, I dunno what they have in other fields.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#443Earlier quoted context omitted.
How many attempts did it take or did you just choose the first one? I have to admit, this is passing my turing test...
Really? I got about halfway through and realized that the comment had no point . If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary. Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....
> Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....
How could we expect it? After 35+ years (BBS and Usenet onward), we've learned that they are often not.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#444"...since Bitcoin" - I hope not. Bitcoin is still a speculative technology, with value based on speculation. It might become useful (or not). NLP is being used right now for practical, commercial tasks. Advancement in NLP is going to serve practical purposes now, with potential for further expansion.
Did you read the article? :)
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#445Earlier quoted context omitted.
I did not guess that it was fake but skipped to the bottom because the article did not seem to be worth reading. It felt like the author was not moving anywhere with their words. I laughed out loud at the reveal that it was written programmatically.
You raise a good point. The Internet has trained us to skim any text that seems pointless or just unsufficiently insightful. So it turns out we have already built up some "mental defences" against GPT-3.
Now we need to make human augmented generative adversarial eyetracking networks that are trained on getting people not to skim.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#446The text generation quality is amazing, but the thing that's blowing my mind is how it can do math problems not seen in the training data. I've also seen a couple demonstrations of people generating simple apps with just a description.
>I've also seen a couple demonstrations of people generating simple apps with just a description.
And yet none of those people have released a demo, even though some said they would, multiple times. I'm still quite sceptical at those demonstration until I get to try it myself.
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#447Earlier quoted context omitted.
Of course, the human can simply lend their name to the robot. And as previously discussed, ending anonymity entrenches the existing power structure.
> Of course, the human can simply lend their name to the robot. That could have consequences for their reputation, though.
Are you suggesting that people found to have misled the public should be .. cancelled?
(Reputation is a lot more controversial and complicated than it sounds)
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#448A lot of people are saying that GPT-3 is a huge step up from GPT-2. This is confusing to me looking at the results in the paper as someone not too familiar with GPT-2. The tasks where GPT-3 scores much higher than GPT-2 are the ones most removed from broad language comprehension / general intelligence: arithmetic and unscrambling tasks (as in: which character does not belong in the string "app$le"). On the other task…
The important thing is that there is qualitative improvement only from quantitive increase. Other software doesn't function like this, switching to a better CPU won't enable new features in Excel. This is not a given, and the fact that there is some proof that it could happen is significant, although it's too early and unsure to really tell. We are nowhere near the border of the amount of resources we can put into a bigger model, what if it just keeps getting better?
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#449Earlier quoted context omitted.
The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a] Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the s…
It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually d…
I am seeing it being used in tons of academic settings, especially with distance learning haha
Re: OpenAI's GPT-3 may be the biggest thing since Bitcoin
#450Earlier quoted context omitted.
Really? I got about halfway through and realized that the comment had no point . If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary. Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....
Also both comments made me suspicious half way through and I scrolled to the bottom to check for a GPT-3 note. Without that note I would definitely have regarded it as incoherent rambling by a human. Especially the second comment can be coherently interpreted with some good will and a cynical view of the humanities and philosophy. The "author" could say that once GPT-3 can write humanities papers it will quickly make…
Who here thinks some Donald Trump's answers were written by an early version of GPT3, designed to produce more bombastic and rambling rhetoric than usual?