How do you go about running a model this large?
GPT-3: Language Models Are Few-Shot Learners
21–30 of 212 posts
Re: GPT-3: Language Models Are Few-Shot Learners
#22How do you go about running a model this large?
Azure
Re: GPT-3: Language Models Are Few-Shot Learners
#23How do you go about running a model this large?
I would hazard a guess that they will release versions that will be a smaller size (they have in the past). But in order to run this, you'd just have to use a cloud provider, first guesses say it'll be 500GB+ of just weights that ideally you want in memory.
Re: GPT-3: Language Models Are Few-Shot Learners
#24How do you go about running a model this large?
Azure
This post may be tongue in cheek, but realistically that's right. I ran the Facebook chatbot that was posted maybe a month ago and didn't get good performance until I was using many Tesla v100 GPUs, which are $8k each. Thankfully modern CSPs like Azure allow renting rigs that can handle this pretty easily.
Re: GPT-3: Language Models Are Few-Shot Learners
#25How do you go about running a model this large?
I would hazard a guess that they will release versions that will be a smaller size (they have in the past). But in order to run this, you'd just have to use a cloud provider, first guesses say it'll be 500GB+ of just weights that ideally you want in memory.
Even the Teslas I use have 256GB memory and those are pretty cutting edge. 500GB in GPU memory is insane.
Re: GPT-3: Language Models Are Few-Shot Learners
#26Earlier quoted context omitted.
Azure
This post may be tongue in cheek, but realistically that's right. I ran the Facebook chatbot that was posted maybe a month ago and didn't get good performance until I was using many Tesla v100 GPUs, which are $8k each. Thankfully modern CSPs like Azure allow renting rigs that can handle this pretty easily.
How much does that cost?
Re: GPT-3: Language Models Are Few-Shot Learners
#27How do you go about running a model this large?
I would hazard a guess that they will release versions that will be a smaller size (they have in the past). But in order to run this, you'd just have to use a cloud provider, first guesses say it'll be 500GB+ of just weights that ideally you want in memory.
Could we bank on the Lottery Ticket Hypothesis, distillation, or other model compression algorithms to make these models smaller?
Re: GPT-3: Language Models Are Few-Shot Learners
#28This part really freaked me out... GPT-2 couldn't do math:
Context → Passage: Saint Jean de Br´ebeuf was a French Jesuit missionary who travelled to New France in 1625. There he worked primarily with the Huron for the rest of his life, except for a few years in France from 1629 to 1633. He learned their language and culture, writing extensively about each to aid other missionaries. In 1649, Br´ebeuf and another missionary were captured when an Iroquois raid took over a Huron village . Together with Huron captives, the missionaries were ritually tortured and killed on March 16, 1649. Br´ebeuf was beatified in 1925 and among eight Jesuit missionaries canonized as saints in the Roman Catholic Church in 1930.
Question: How many years did Saint Jean de Br´ebeuf stay in New France before he went back to France for a few years?
Answer: Completion → 4
Re: GPT-3: Language Models Are Few-Shot Learners
#29How do you go about running a model this large?
Realistically you'd be able to train up to 760M param models. For that you'd need 32gb+ VRAM GPUs which I think AWS might have. You can try iwth 16gb VRAM GPUs, but you would need to figure out FP16.
https://github.com/shawwn is doing some work in the GPT-2 space including using TPUs instead -- which has given him pretty good results.
Re: GPT-3: Language Models Are Few-Shot Learners
#30This part really freaked me out... GPT-2 couldn't do math: Context → Passage: Saint Jean de Br´ebeuf was a French Jesuit missionary who travelled to New France in 1625. There he worked primarily with the Huron for the rest of his life, except for a few years in France from 1629 to 1633. He learned their language and culture, writing extensively about each to aid other missionaries. In 1649, Br´ebeuf and another missi…
T5 could, and well.