T0* – Series of encoder-decoder models trained on a large set of different tasks
1–10 of 163 posts
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#2ArXiv link to the paper: https://arxiv.org/abs/2110.08207
GitHub repo: https://github.com/bigscience-workshop/promptsource
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#3[Disclaimer: I am an author of the above paper and played a rather minimal role. I am also a prominent member of EleutherAI.]
"Instruction-tuning" is clearly in the air. Simultaneous work at Google (released less than two weeks ago) on a model they call FLAN can be found here: https://ai.googleblog.com/2021/10/introducing-flan-more-gene...
EleutherAI attempted to do something similar several months ago, but didn't succeed: https://blog.eleuther.ai/tuning-on-eval-harness/
A careful analysis of the similarities and differences between the three approaches would be likely highly beneficial to the community.
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#4[Disclaimer: I am an author of the above paper and played a rather minimal role. I am also a prominent member of EleutherAI.] "Instruction-tuning" is clearly in the air. Simultaneous work at Google (released less than two weeks ago) on a model they call FLAN can be found here: https://ai.googleblog.com/2021/10/introducing-flan-more-gene... EleutherAI attempted to do something similar several months ago, but didn't su…
Just in case this question isn't to far out of your way.
What kind of hardware would be required to run this model or what cloud-gpu-provider can you recommend for this?
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#5I'm not familiar with the current state of the art language models, so please bear with me for asking: What's the catch here? Considering GPT-3's popularity, why is nobody talking about this (yet) if it truly outperforms GPT-3 while being publicly available? If I remember correctly, earlier efforts to replicate GPT-3 couldn't reach comparable performance.
Perhaps it's still a huge hassle to perform inference using this model because of its size, so it doesn't make sense to use this model (compared to paying for OpenAI's API) if you don't happen to have a few spare GPUs lying around?
Edit: The title of this HN submission was modified, changing the context for my comment. Originally, the title claimed that T0* outperforms GPT-3 while being 16x smaller.
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#6[Disclaimer: I am an author of the above paper and played a rather minimal role. I am also a prominent member of EleutherAI.] "Instruction-tuning" is clearly in the air. Simultaneous work at Google (released less than two weeks ago) on a model they call FLAN can be found here: https://ai.googleblog.com/2021/10/introducing-flan-more-gene... EleutherAI attempted to do something similar several months ago, but didn't su…
Just want to say thanks for taking the time to put the model on HuggingFace! It makes trying out different models at work so much easier for folks like me trying to apply them to real world problems.
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#7It's funny how much of the page they dedicate to talking about mitigations of sexism and bias. Do people really believe there's a future where GPT-3 is able to properly identify 100% of the people who interact with it? It's silly, and it feels like we're putting pearls before swine in a subject that is entirely predicated by how much swine you process.
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#8gpt3 is good for large generation tasks and for "true" zero shotting (as much as this is possible). people know this. this is a weird title
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#9First time I hear about BigScience - very interesting.
Re: T0* – Series of encoder-decoder models trained on a large set of different tasks
#10I tried querying it about the airspeed velocity of an unladen swallow and it answered "0 km/h". Still needs some work I guess :)