Wonder what the split is between Russian and English in the model?
YaLM-100B: Pretrained language model with 100B parameters
41–50 of 666 posts
Re: YaLM-100B: Pretrained language model with 100B parameters
#42Earlier quoted context omitted.
What about 250gb of ram and use a cpu ?
$ dd if=/dev/zero of=/swapfile bs=1G count=250 status=progress $ chmod 600 /swapfile $ mkswap -U clear /swapfile $ swapon /swapfile
Is there a reason why it is required to fill the swapfile with zeroes here? Normally you'd see something like "dd of=/swapfile bs=1G seek=3 count=0", creating a file of size 3G but with no space allocated (yet). It's much quicker to complete the setup this way.
Re: YaLM-100B: Pretrained language model with 100B parameters
#43Earlier quoted context omitted.
Coming from Russia doesn't mean you agree with government policy. If you saw people get arrested as soon as they start protesting, what would you do?
But as a company, they do. They are filtering alternative media from their search.
Genuinely curious: which media outlets are banned from Yandex Search? I just tried Meduza for example, comes up in search just fine.
Re: YaLM-100B: Pretrained language model with 100B parameters
#44I have huge respect for developers at Yandex. It's kind of sad that achievements like these are tainted by the fact that they come from Russia (and I speak as a Ukrainian). I wonder if the permissive license is able to mitigate that.
Are American developers’ achievement tainted by the fact they come from the United States?
Underrated comment, considering the history of NATO expansion, color revolutions and the hundreds of thousands killed in pursuit of reckless ideological overt or covert warfare.
Re: YaLM-100B: Pretrained language model with 100B parameters
#45I have huge respect for developers at Yandex. It's kind of sad that achievements like these are tainted by the fact that they come from Russia (and I speak as a Ukrainian). I wonder if the permissive license is able to mitigate that.
The achievements aren’t in any way tainted by their nationality, citizenship, sex, sexual orientation, age, etc.
Well... I'm sorry if I reach for the reductio at Hitlerum, but any achievements Nazi scientists might have reached in concentration camps are definitely tainted. Similarly, achievements in the field of online consumer analysis in a country where consumer-privacy protections are nonexistent, surely should be considered tainted...?
Re: YaLM-100B: Pretrained language model with 100B parameters
#46I hope one day it will be possible to run this kind of models at home.
I think that unlikely. Barring some breakthrough that takes us beyond the limits of silicon.
Re: YaLM-100B: Pretrained language model with 100B parameters
#47Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.
Can Apple Silicone's unified memory be an answer?
Re: YaLM-100B: Pretrained language model with 100B parameters
#48I have huge respect for developers at Yandex. It's kind of sad that achievements like these are tainted by the fact that they come from Russia (and I speak as a Ukrainian). I wonder if the permissive license is able to mitigate that.
Are American developers’ achievement tainted by the fact they come from the United States?
Some of them, like the invasive analysis tools that exploit the Patriot Act, yes for sure.
Re: YaLM-100B: Pretrained language model with 100B parameters
#49Quoted post unavailable.
Who controls Google? What did Google do to stop the inasion of Iraq? Will Google take responsibility for silent support od war in Iraq?
Your comparison fails a test of facts. Yandex actively censors any perspective not approved by the Kremlin. Google does not do anything comparable to this.
Re: YaLM-100B: Pretrained language model with 100B parameters
#50I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…
> I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers.
Isn’t that already the case? Sure, it costs $60K, but that is accessible to a surprisingly large minority, considering the potency of this software.