Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

131–140 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#131
post #6

I have huge respect for developers at Yandex. It's kind of sad that achievements like these are tainted by the fact that they come from Russia (and I speak as a Ukrainian). I wonder if the permissive license is able to mitigate that.

What percentage of American inventions and scientific developments post WW2 were led or influenced by former Nazi scientists?

Re: YaLM-100B: Pretrained language model with 100B parameters

#132
post #88

Earlier quoted context omitted.

log(300) / log(2) = only 8.2 doublings away. That's near future material.

Maybe at 90s hardware growth rates, but not now.

The dream of the 90s is alive in the GPU market: https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-fl...

Moore's law didn't stop, just Dennard scaling. Expect graphics and AI to continue to improve radically in performance/price, while more ordinary workloads see only modest improvements.

Re: YaLM-100B: Pretrained language model with 100B parameters

#134
post #71

Earlier quoted context omitted.

It could be possible with analog chips. I.e. ones that Mythic works on.

I'm not sure why you got downvoted. Yes, ASICs (either analog or digital) that have some model hardcoded in would probably make it feasible, but it won't be programmable which is the interesting part.

Totally not my field, but why wouldn't they be programmable? Analog FPGA's already exist.

Re: YaLM-100B: Pretrained language model with 100B parameters

#136
post #45
post #8

Earlier quoted context omitted.

The achievements aren’t in any way tainted by their nationality, citizenship, sex, sexual orientation, age, etc.

Well... I'm sorry if I reach for the reductio at Hitlerum, but any achievements Nazi scientists might have reached in concentration camps are definitely tainted. Similarly, achievements in the field of online consumer analysis in a country where consumer-privacy protections are nonexistent, surely should be considered tainted...?

And Yandex's AI work got helped by the Russian invasion of Ukraine how, exactly? Did they train the bots on Ukrainian captives first?

Re: YaLM-100B: Pretrained language model with 100B parameters

#137

Earlier quoted context omitted.

What about 250gb of ram and use a cpu ?

$ dd if=/dev/zero of=/swapfile bs=1G count=250 status=progress $ chmod 600 /swapfile $ mkswap -U clear /swapfile $ swapon /swapfile

If you bother to set the permissions, I suggest to do it in a way that doesn't leave a time window during which it still is unprotected (note that non-priviledged processes just need to open the file during that window; they can keep reading even after your chmod has been run). Also, not sure what the point of `-U clear` was, that's setting the uuid for the swap, better leave it at the default random one?

    $ ( umask 077; dd if=/dev/zero of=/swapfile bs=1G count=250 status=progress )
    $ mkswap /swapfile
    $ swapon /swapfile

Re: YaLM-100B: Pretrained language model with 100B parameters

#138
Side note: Yandex search is awesome, and I really hope they stay alive forever. It's the only functional image search nowadays, after our Google overlords neutered their own product out of fear over lawyers/regulation and a disdain for power users.

You can't even search for images "before:date" in Google anymore.

Re: YaLM-100B: Pretrained language model with 100B parameters

#140
post #9

Earlier quoted context omitted.

Coming from Russia doesn't mean you agree with government policy. If you saw people get arrested as soon as they start protesting, what would you do?

I am asking this question myself for 120 days and I still don't have an answer.

Maybe not be "apolitical" for 20 years that led to this?

Russian IT and media were showered with relatively high wages to stay quiet while last semblance of elections was finally destroyed, independent media was took over by pro-Putin oligarchs and activists were crushed or murdered.

As was the sarcastic saying coined recently, "if you are apolitical then bullets don't hit you".

Post reply on HN