I have huge respect for developers at Yandex. It's kind of sad that achievements like these are tainted by the fact that they come from Russia (and I speak as a Ukrainian). I wonder if the permissive license is able to mitigate that.
YaLM-100B: Pretrained language model with 100B parameters
131–140 of 666 posts
Re: YaLM-100B: Pretrained language model with 100B parameters
#132Earlier quoted context omitted.
log(300) / log(2) = only 8.2 doublings away. That's near future material.
Maybe at 90s hardware growth rates, but not now.
Moore's law didn't stop, just Dennard scaling. Expect graphics and AI to continue to improve radically in performance/price, while more ordinary workloads see only modest improvements.
Re: YaLM-100B: Pretrained language model with 100B parameters
#133Re: YaLM-100B: Pretrained language model with 100B parameters
#134Earlier quoted context omitted.
It could be possible with analog chips. I.e. ones that Mythic works on.
I'm not sure why you got downvoted. Yes, ASICs (either analog or digital) that have some model hardcoded in would probably make it feasible, but it won't be programmable which is the interesting part.
Re: YaLM-100B: Pretrained language model with 100B parameters
#135Quoted post unavailable.
Re: YaLM-100B: Pretrained language model with 100B parameters
#136Earlier quoted context omitted.
The achievements aren’t in any way tainted by their nationality, citizenship, sex, sexual orientation, age, etc.
Well... I'm sorry if I reach for the reductio at Hitlerum, but any achievements Nazi scientists might have reached in concentration camps are definitely tainted. Similarly, achievements in the field of online consumer analysis in a country where consumer-privacy protections are nonexistent, surely should be considered tainted...?
Re: YaLM-100B: Pretrained language model with 100B parameters
#137Earlier quoted context omitted.
What about 250gb of ram and use a cpu ?
$ dd if=/dev/zero of=/swapfile bs=1G count=250 status=progress $ chmod 600 /swapfile $ mkswap -U clear /swapfile $ swapon /swapfile
$ ( umask 077; dd if=/dev/zero of=/swapfile bs=1G count=250 status=progress )
$ mkswap /swapfile
$ swapon /swapfileRe: YaLM-100B: Pretrained language model with 100B parameters
#138You can't even search for images "before:date" in Google anymore.
Re: YaLM-100B: Pretrained language model with 100B parameters
#139Re: YaLM-100B: Pretrained language model with 100B parameters
#140Earlier quoted context omitted.
Coming from Russia doesn't mean you agree with government policy. If you saw people get arrested as soon as they start protesting, what would you do?
I am asking this question myself for 120 days and I still don't have an answer.
Russian IT and media were showered with relatively high wages to stay quiet while last semblance of elections was finally destroyed, independent media was took over by pro-Putin oligarchs and activists were crushed or murdered.
As was the sarcastic saying coined recently, "if you are apolitical then bullets don't hit you".