No post body was provided.
YaLM-100B: Pretrained language model with 100B parameters
31–40 of 666 posts
Re: YaLM-100B: Pretrained language model with 100B parameters
#32I hope one day it will be possible to run this kind of models at home.
If your disk has enough space to store the model, I think in theory you could run them, using the disk to store states. But it will be slow. I'm not sure how slow though, and also if anyone has implemented this. It actually should not be too difficult.
Re: YaLM-100B: Pretrained language model with 100B parameters
#33Re: YaLM-100B: Pretrained language model with 100B parameters
#34Quoted post unavailable.
Who controls Google? What did Google do to stop the inasion of Iraq? Will Google take responsibility for silent support od war in Iraq?
Re: YaLM-100B: Pretrained language model with 100B parameters
#35Earlier quoted context omitted.
Way too slow on CPU unfortunately But this does make me wonder if there's any way to allow a graphics card to use regular RAM in a fast way? AFAIK built-in GPU's inside CPU's can but those GPU's are not powerful enough
I thought cuda had a unified memory system? Maybe I misunderstood
Unified memory exists, but it's not a magic bullet. If a page is accessed that doesn't reside on device memory (i.e. on the GPU), a memcpy is issued to fetch the page from main RAM. While the programming model is nicer, it doesn't fundamentally change the fact that you need to constantly swap data out to main RAM and while not as bad as loading it from the SSD or HDD, that's still quite slow.
Integrated GPUs that use a portion of system memory are an exception to this and do not require memcpys when using unified memory. However, I'm not aware of any powerful iGPUs from Nvidia these days.
Re: YaLM-100B: Pretrained language model with 100B parameters
#36I have huge respect for developers at Yandex. It's kind of sad that achievements like these are tainted by the fact that they come from Russia (and I speak as a Ukrainian). I wonder if the permissive license is able to mitigate that.
Are American developers’ achievement tainted by the fact they come from the United States?
Re: YaLM-100B: Pretrained language model with 100B parameters
#37Earlier quoted context omitted.
Coming from Russia doesn't mean you agree with government policy. If you saw people get arrested as soon as they start protesting, what would you do?
But as a company, they do. They are filtering alternative media from their search.
Yandex is under full control of Russian government. Pretty sure FSB can access anything.
Re: YaLM-100B: Pretrained language model with 100B parameters
#38Earlier quoted context omitted.
"We want to punish you" and "We don't want to make business with you" are two very different things IMHO.
If you go out of your way to not do business with someone, in order to cripple them economically, then it is a punishment.
If you want to define punishment like that, it is your call.
In my opinion it is not punishment to stop a relationship if the basis of that relationship was destroyed deliberately by one side.
Re: YaLM-100B: Pretrained language model with 100B parameters
#39For those of us without 200GB of GPU RAM available... How possible is it to do inference loading it from SSD? Would you have to scan through all 200GB of data once per character generated? That doesn't actually sound too painful - 1 minute per character seems kinda okay. And I guess you can easily do lots of data parallelism, so you can get 1 minute per character on lots of inputs and outputs at the same time.
These models are not character-based, but token-based. The problem with CPU inference is the need for random access to 250 GiB of parameters, meaning immense paging and orders of magnitude slower than normal CPU operation.
I wonder how bad it comes out with something like Optane?
Re: YaLM-100B: Pretrained language model with 100B parameters
#40Quoted post unavailable.
Please ask the model to answer these questions.