AI training method exceeds GPT-3 performance with fewer parameters
1–10 of 82 posts
Re: AI training method exceeds GPT-3 performance with fewer parameters
#2> although [the new technique] produced better results for NLP benchmarks, GPT-3 appeared more flexible.
Re: AI training method exceeds GPT-3 performance with fewer parameters
#3GPT-3 is also being used to make hedge funds money by generating investing themes https://medium.com/@492727ZED/dexamethasone-announcement-cou...
Re: AI training method exceeds GPT-3 performance with fewer parameters
#4I'm not surprised, a space with 175B parameters will be very sparse.
Re: AI training method exceeds GPT-3 performance with fewer parameters
#5GPT3's strength is on language generation, so using *GLUE for evaluating it (where encoder type models are just better) and claiming to have 99.9% less parameters is sensationalism.
Re: AI training method exceeds GPT-3 performance with fewer parameters
#6GPT3's strength is on language generation, so using *GLUE for evaluating it (where encoder type models are just better) and claiming to have 99.9% less parameters is sensationalism.
Yes, it can do simple yes/no questions but can it write dad jokes like GPT-3?
Re: AI training method exceeds GPT-3 performance with fewer parameters
#7GPT3's strength is on language generation, so using *GLUE for evaluating it (where encoder type models are just better) and claiming to have 99.9% less parameters is sensationalism.
That's kind of like saying that this super cheap SUV from 2003 is a better off-roader than the latest Ferrari. Like, true statement, but vacuous none the less.
Re: AI training method exceeds GPT-3 performance with fewer parameters
#8If you want to read the source, I recommend skipping to their follow-up paper (same authors): https://arxiv.org/abs/2009.07118
Edit: tangentially related but for those who like to have a glance from their phone, arxiv-vanity is great instead of squinting at a pdf: https://www.arxiv-vanity.com/papers/2009.07118/
Re: AI training method exceeds GPT-3 performance with fewer parameters
#9I just wish I had the vocabulary to truly appreciate what's going on here.
Re: AI training method exceeds GPT-3 performance with fewer parameters
#10GPT3's strength is on language generation, so using *GLUE for evaluating it (where encoder type models are just better) and claiming to have 99.9% less parameters is sensationalism.
Yes, it can do simple yes/no questions but can it write dad jokes like GPT-3?
For me another question is crucial: is it open (i.e. is there an open source reference implementation) or closed like the so-called "OpenAI" products/services?