I'm not familiar with the current state of the art language models, so please bear with me for asking: What's the catch here? Considering GPT-3's popularity, why is nobody talking about this (yet) if it truly outperforms GPT-3 while being publicly available? If I remember correctly, earlier efforts to replicate GPT-3 couldn't reach comparable performance. Perhaps it's still a huge hassle to perform inference using th…
I'm not familiar with the current state of the art language models, so please bear with me for asking: What's the catch here? Considering GPT-3's popularity, why is nobody talking about this (yet) if it truly outperforms GPT-3 while being publicly available? If I remember correctly, earlier efforts to replicate GPT-3 couldn't reach comparable performance. Perhaps it's still a huge hassle to perform inference using th…
The paper on this new model seems to have been published just 3 days ago, so I think it takes time for the wider community to verify their claims and for this to gain wider acceptance.
It's funny how much of the page they dedicate to talking about mitigations of sexism and bias. Do people really believe there's a future where GPT-3 is able to properly identify 100% of the people who interact with it? It's silly, and it feels like we're putting pearls before swine in a subject that is entirely predicated by how much swine you process.
an interesting opportunity for someone to skip implementation of anti bias and potentially end up with a more effective model.
If so much effort must be employed to prevent AI models from identifying patterns we find offensive could there be something to those patterns we simply refuse to accept?
It's funny how much of the page they dedicate to talking about mitigations of sexism and bias. Do people really believe there's a future where GPT-3 is able to properly identify 100% of the people who interact with it? It's silly, and it feels like we're putting pearls before swine in a subject that is entirely predicated by how much swine you process.
Making sure that NLP algorithms are unbiased is important not just from a social justice perspective, but from a perspective of how useful the algorithms are. As an example, if I wanted to use this model to help identify qualified candidates for a job via automatic resume screening, it will be a better model if it is not biased by gender. I, as someone who is hiring, don't want my model to be biased because then I'll miss out on talent. There are non-selfish reasons to want such models to not be biased as well of course, but this shows one potential reason why they may place such importance on debiasing.
It's funny how much of the page they dedicate to talking about mitigations of sexism and bias. Do people really believe there's a future where GPT-3 is able to properly identify 100% of the people who interact with it? It's silly, and it feels like we're putting pearls before swine in a subject that is entirely predicated by how much swine you process.
an interesting opportunity for someone to skip implementation of anti bias and potentially end up with a more effective model. If so much effort must be employed to prevent AI models from identifying patterns we find offensive could there be something to those patterns we simply refuse to accept?
I think that you don't quite understand how these models pick up these biases. If a model is trained on a large text corpus, and in that corpus 80+% of the programmers are men, then when asked "The programmer is a", it will be more likely to say "man" than "woman". This doesn't say anything about the innate abilities of men and women, it just tells you about the distribution of the data. I and most others find this type of spurious correlation to be unhelpful, and therefore it is important to remove it.
I'm not familiar with the current state of the art language models, so please bear with me for asking: What's the catch here? Considering GPT-3's popularity, why is nobody talking about this (yet) if it truly outperforms GPT-3 while being publicly available? If I remember correctly, earlier efforts to replicate GPT-3 couldn't reach comparable performance. Perhaps it's still a huge hassle to perform inference using th…
(author here)
The paper/model/code was just made public today. This may be why no one is talking about it yet.
Regarding whether the size is a hassle: It's possible to run inference on a single Google Cloud TPU v3-8 device or on a server with 4x 32GB v100 GPUs. Hugging Face also has an inference API for any model on the Hub: https://api-inference.huggingface.co/docs/python/html/index....
The hosted demo has the default query, "How many hydrogen atoms are in a water molecule?" It said "two". I asked it, "How many oxygen atoms are in a water molecule?". It said "two".
There are apparently also two carbon atoms in a water molecule. But only one Donald Trump.
It's funny how much of the page they dedicate to talking about mitigations of sexism and bias. Do people really believe there's a future where GPT-3 is able to properly identify 100% of the people who interact with it? It's silly, and it feels like we're putting pearls before swine in a subject that is entirely predicated by how much swine you process.
It's literally the last section of the page, just before the citations, and it's only a few paragraphs + two tables to show the model's performance on industry standard benchmarks.