Quoted post unavailable.
Who controls Google? What did Google do to stop the inasion of Iraq? Will Google take responsibility for silent support od war in Iraq?
[flagged]
YaLM-100B: Pretrained language model with 100B parameters
51–60 of 666 posts
Re: YaLM-100B: Pretrained language model with 100B parameters
#52For those of us without 200GB of GPU RAM available... How possible is it to do inference loading it from SSD? Would you have to scan through all 200GB of data once per character generated? That doesn't actually sound too painful - 1 minute per character seems kinda okay. And I guess you can easily do lots of data parallelism, so you can get 1 minute per character on lots of inputs and outputs at the same time.
These models are not character-based, but token-based. The problem with CPU inference is the need for random access to 250 GiB of parameters, meaning immense paging and orders of magnitude slower than normal CPU operation. I wonder how bad it comes out with something like Optane?
It's not really random access. I bet the graph can be pipelined such that you can keep a "horizontal cross-section" of the graph in memory all the time, and you scan through the parameters from top to bottom in the graph.
Re: YaLM-100B: Pretrained language model with 100B parameters
#53Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.
Wondering if Apple Silicon will bring arge amounts of unified main memory with high bandwidth to the masses?
The Mac Studio maxes out at 128GB currently for around $5K, so 256GB isn't that far out and might work with the ~200GB Yandex says is required.
Re: YaLM-100B: Pretrained language model with 100B parameters
#54Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.
Take a look at Apple's M1 Max, a lot of fast unified memory. No idea how useful though
Re: YaLM-100B: Pretrained language model with 100B parameters
#55Earlier quoted context omitted.
What about 250gb of ram and use a cpu ?
Way too slow on CPU unfortunately But this does make me wonder if there's any way to allow a graphics card to use regular RAM in a fast way? AFAIK built-in GPU's inside CPU's can but those GPU's are not powerful enough
Slow, but is it still practical, like taking minutes to generate few words ca still be useful for testing or on certain low usage use-cases ?
Re: YaLM-100B: Pretrained language model with 100B parameters
#56Earlier quoted context omitted.
The achievements aren’t in any way tainted by their nationality, citizenship, sex, sexual orientation, age, etc.
Well... I'm sorry if I reach for the reductio at Hitlerum, but any achievements Nazi scientists might have reached in concentration camps are definitely tainted. Similarly, achievements in the field of online consumer analysis in a country where consumer-privacy protections are nonexistent, surely should be considered tainted...?
I suppose the question should be: did the malevolence help in reaching the results?
Re: YaLM-100B: Pretrained language model with 100B parameters
#57Earlier quoted context omitted.
They're just following the law of their host country, like DuckDuckGo and Google have to... What's the alternative? Open rebellion against the state?
They've started to do so before the law was adjusted
Like Facebook, Twitter and every other forum did?
I wonder if there were any consequences if they didn't.
Re: YaLM-100B: Pretrained language model with 100B parameters
#58I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…
It could be possible with analog chips. I.e. ones that Mythic works on.
Re: YaLM-100B: Pretrained language model with 100B parameters
#59Earlier quoted context omitted.
These models are not character-based, but token-based. The problem with CPU inference is the need for random access to 250 GiB of parameters, meaning immense paging and orders of magnitude slower than normal CPU operation. I wonder how bad it comes out with something like Optane?
It's not really random access. I bet the graph can be pipelined such that you can keep a "horizontal cross-section" of the graph in memory all the time, and you scan through the parameters from top to bottom in the graph.
I wonder if you can't do that LSH trick to turn it into a sparse matrix problem and run it on CPU that way.
Re: YaLM-100B: Pretrained language model with 100B parameters
#60I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…
> I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Isn’t that already the case? Sure, it costs $60K, but that is accessible to a surprisingly large minority, considering the potency of this software.
...what? 60 thousand dollars for a dedicated computer that you can't use is not everyone, not on their own computers, and is also a crazy large amount of money for nearly everyone. Sure there are some that could, but that's not what I said.