Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

191–200 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#191

Post by trasz »

skrebbel wrote: post #136
toyg wrote: post #45

Earlier quoted context omitted.

Well... I'm sorry if I reach for the reductio at Hitlerum, but any achievements Nazi scientists might have reached in concentration camps are definitely tainted. Similarly, achievements in the field of online consumer analysis in a country where consumer-privacy protections are nonexistent, surely should be considered tainted...?

And Yandex's AI work got helped by the Russian invasion of Ukraine how, exactly? Did they train the bots on Ukrainian captives first?

I'd guess they've been getting a whole lot more traffic lately, thanks to ban on Western services.

Re: YaLM-100B: Pretrained language model with 100B parameters

#192

Post by simonh »

alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

It's more likely, if not inevitable that these things will become ubiquitously available remotely, like Siri and Alexa. It's access that's important, not hosting.

Re: YaLM-100B: Pretrained language model with 100B parameters

#193

Post by pastacacioepepe »

lotusmars wrote: post #144
trhway wrote: post #64

Earlier quoted context omitted.

> Western companies and government pulling out of Russia you can't really blame the companies for not wanting to be associated in any way with a nazist regime genociding a neighboring country. And Starbucks or Mercedes pulling out of Russia isn't punishment. It is freedom of association and economic activity. Russians are whining about "punishment" because they have no idea about freedom, and that is them getting a b…

It was hilarious when people from Moscow wrote "they took away our ability to buy Chanel bags, so much for European tolerance!" Not even seeing the irony. What did they expect in response to bombing, pillaging, mass rape? Friendly hug?

If I counted every time a NATO member committed worse atrocities but wasn't held accountable at all I'd probably stop after Ukraine is conquered.

What did they expect? Probably indifference, same reaction to any war crime committed by the west so far.

Re: YaLM-100B: Pretrained language model with 100B parameters

#194

Post by londons_explore »

edf13 wrote: post #41

Wonder what the split is between Russian and English in the model?

Open the vocab file (from the script in the download directory) and you can get a pretty good idea.

Looks to be approximately 50/50 from my random scrolling through the list.

Re: YaLM-100B: Pretrained language model with 100B parameters

#195

Post by memorable »

pembrook wrote: post #138

Side note: Yandex search is awesome, and I really hope they stay alive forever. It's the only functional image search nowadays, after our Google overlords neutered their own product out of fear over lawyers/regulation and a disdain for power users. You can't even search for images "before:date" in Google anymore.

I agree with this. When I am still addicted to porn, Yandex Image is the only one that seems to find relevant and useful links.

Re: YaLM-100B: Pretrained language model with 100B parameters

#196

Post by trasz »

pastacacioepepe wrote: post #154
layer8 wrote: post #79

Earlier quoted context omitted.

The purpose is deterrence, not punishment.

And yet sanctions never work as a deterrent. Cuba is still socialist after 60 years of sanctions. Great deterrent! No, sanctions just punish generation after generation of innocent people and serve no other purpose. If you still mantain that the purpose is deterrence, then you must be a fool or worse, since it never works! Can't you learn from the past? Edit: Before you even dig out some article like this: https://ww…

And the fact that Russia is not capable of manufacturing tanks anymore has nothing to do whatsoever with sanctions?

Re: YaLM-100B: Pretrained language model with 100B parameters

#197

Post by uniqueuid »

taink wrote: post #182
uniqueuid wrote: post #104

Earlier quoted context omitted.

Nitpick: This uses 8x A100 which are at least $10k a piece to my knowledge. Add in the computer and you're closer to $100k.

I believe you're confusing the amount of A100 graphics cards used to train the model (the cluster was actually made up of 800 A100s), and the amount you need to run the model : > The model [...] is supposed to run on multiple GPUs with tensor parallelism. > It was tested on 4 (A100 80g) and 8 (V100 32g) GPUs, [but should work] with ≈200GB of GPU memory. I don't know what the price of a V100 is, but given $10k a piece…

The $10k price is for an A100 with 40GB ram, so you need 8 of those. If you can get your hands on the 80GB variant, 4 are enough.

Also, if you want to have a machine with eight of these cards, it will need to be a pretty high-spec rack-mounted or large tower. To feed these GPUs, you will want to have a decent amount of PCIe-4 lanes, meaning EPYC are the logical choice. So that's $20k for an AMD EPYC server with at least 1.6kw PSUs etc etc.

Re: YaLM-100B: Pretrained language model with 100B parameters

#198

Post by ushakov »

bgandrew wrote: post #155
kome wrote: post #139

I agree, yandex is a great search engine

no it's not. they straight up serve kremlin, promoting kremlin fake news and silencing russian opposition (not much to silence but still). they can have whatever functionality they like, I still won't use it in billion years.

can you name any Russian company that doesn't?

obeying to Kremlin is just an aspect of running business in Russia

the only option would be not to operate in Russia at all. Yandex can't do this, because their audience is primarily in Russia

Re: YaLM-100B: Pretrained language model with 100B parameters

#199

Post by memorable »

huma wrote: post #147
keewee7 wrote: post #133

Quoted post unavailable.

Not for being Russians, but for active participation in censorship by tweaking their news aggregation to show only hand picked government approved sources

Is there a source for this? I'm curious.

Re: YaLM-100B: Pretrained language model with 100B parameters

#200

Post by The_Colonel »

skrebbel wrote: post #136
toyg wrote: post #45

Earlier quoted context omitted.

Well... I'm sorry if I reach for the reductio at Hitlerum, but any achievements Nazi scientists might have reached in concentration camps are definitely tainted. Similarly, achievements in the field of online consumer analysis in a country where consumer-privacy protections are nonexistent, surely should be considered tainted...?

And Yandex's AI work got helped by the Russian invasion of Ukraine how, exactly? Did they train the bots on Ukrainian captives first?

It's the other way round. It's quite possible that e.g. this model helped to spread Kremlin propaganda in various discussion boards.
Post reply on HN