Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

251–260 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#251

Post by cpursley »

lotusmars wrote: post #144
trhway wrote: post #64

Earlier quoted context omitted.

> Western companies and government pulling out of Russia you can't really blame the companies for not wanting to be associated in any way with a nazist regime genociding a neighboring country. And Starbucks or Mercedes pulling out of Russia isn't punishment. It is freedom of association and economic activity. Russians are whining about "punishment" because they have no idea about freedom, and that is them getting a b…

It was hilarious when people from Moscow wrote "they took away our ability to buy Chanel bags, so much for European tolerance!" Not even seeing the irony. What did they expect in response to bombing, pillaging, mass rape? Friendly hug?

Just want to point out the mass rape claims were fabricated. Ukraine even fired Lyudmila Denisova over the ordeal:

https://greekcitytimes.com/2022/06/01/ukrainian-official-fir...

Re: YaLM-100B: Pretrained language model with 100B parameters

#252

Post by [deleted] »

OmicronCeti wrote: post #226
yonixw wrote: post #173

Earlier quoted context omitted.

Bad comment. First, can you please name the founder? Because according to wiki Ilya Segalovich never lived in Israel and Arkady_Volozh lives in Tel Aviv (Not a settlement). Both Jewish, so why present it as some "must-be-hidden-cause" connection with Israel? Also, nothing shady from Israel side in term of sanctions. They have a large Jewish community in both Russia and Ukraine and need to be on good term with both to…

>Elena Bunina, who is Jewish, is stepping down from her role as CEO of Yandex LLC, 'Russia's Google,' amid the war in Ukraine. Sources confirm she is in Israel and has no intention of returning to Russia https://www.haaretz.com/israel-news/tech-news/2022-04-06/ty-...

[deleted]

Re: YaLM-100B: Pretrained language model with 100B parameters

#254

Post by toxik »

amelius wrote: post #52
toxik wrote: post #39

Earlier quoted context omitted.

These models are not character-based, but token-based. The problem with CPU inference is the need for random access to 250 GiB of parameters, meaning immense paging and orders of magnitude slower than normal CPU operation. I wonder how bad it comes out with something like Optane?

It's not really random access. I bet the graph can be pipelined such that you can keep a "horizontal cross-section" of the graph in memory all the time, and you scan through the parameters from top to bottom in the graph.

Fair point, but you’ll still be bounded by disk read speed on an SSD. The access pattern itself matters less than the read cache being << the parameter set size.

Re: YaLM-100B: Pretrained language model with 100B parameters

#255

Post by daniel-cussen »

simonh wrote: post #178
ok123456 wrote: post #156

Earlier quoted context omitted.

Quoted post unavailable.

It's not that Russia Bad, it's that if you know a search engine will serve you censored, biased results that makes it an unreliable search engine.

Compared to whom!? Who will serve you uncensored, unbiased results!? Like run your own crawler, dude grep, go to the library, come on!

Re: YaLM-100B: Pretrained language model with 100B parameters

#256

Post by brailsafe »

yonixw wrote: post #242
keewee7 wrote: post #237

Earlier quoted context omitted.

Many people consider all of Israel to be an illegitimate Western colonial settler state in the Middle East.

Then they have some explaining to do. As they are very wrong, from a historical/archaeological point view [1]. [1] https://en.wikipedia.org/wiki/History_of_the_Jews_and_Judais...

> Then they have some explaining to do. As they are very wrong, from a historical point view [1]

Kind of explains the general vibe of relations between everyone in that area historically

Re: YaLM-100B: Pretrained language model with 100B parameters

#257

Post by lotusmars »

cpursley wrote: post #251
lotusmars wrote: post #144

Earlier quoted context omitted.

It was hilarious when people from Moscow wrote "they took away our ability to buy Chanel bags, so much for European tolerance!" Not even seeing the irony. What did they expect in response to bombing, pillaging, mass rape? Friendly hug?

Just want to point out the mass rape claims were fabricated. Ukraine even fired Lyudmila Denisova over the ordeal: https://greekcitytimes.com/2022/06/01/ukrainian-official-fir...

No, there are credible and quite horrifying reports[1]. Also some of the actual rapists were found out and victimes stepped forward. Please don't spew Russian propaganda.

[1] https://meduza.io/en/feature/2022/04/18/i-can-do-whatever-i-...

Re: YaLM-100B: Pretrained language model with 100B parameters

#258

Post by jeroenhd »

lukestateson wrote: post #168

[flagged]

Russian company follows Russian propaganda rules, that's hardly news. It's pretty clear that concepts like "free press" and "freedom of information" aren't compatible with the Russian regime and expecting such features from a company operating mostly in Russia is kind of pointless. It should be obvious that anything Yandex (or any company targeting Russia, really) should be met with a good deal of scepticism. Companies like Yandex and Baidu can still deliver usable research, though, as long as you realise with what kind of perspective their code was written and their algorithm trained.

In a similar vain, Microsoft has censored "tank man" from their image search (and that of all their image search customers, such as DuckDuckGo). Google is a more transparent about their censorship, usually showing a link or explanation why they remove certain information at the bottom of the page, but it still reflects the values of western civilisation, for example by delisting Russian propaganda such as RT.

These biases are everywhere in all research into this field. The Russian situation is obviously worse than that in many other countries, but you should never forget the bias that AI models from free countries have been trained with either.

Re: YaLM-100B: Pretrained language model with 100B parameters

#259

Post by natly »

Planktonne wrote: post #185
Comevius wrote: post #80

Earlier quoted context omitted.

That's definitely the future, personalized entertainment and social interactions will be big. I could watch a movie made for me, and discuss it with a bunch of chat bots. The future will be bubbly as hell, people will be decaying in their safe places as the hellscape rages on outside.

> I could watch a movie made for me We're a long, long way from this. Stringing words/images together into a coherent sequence is arguably the easy bit of creating novels/films, and computers still lag a long way behind humans in this regard. Structuring a narrative is a harder, subtler step. Our most advanced ML solutions are improving rapidly, but often struggle with coherence over a single paragraph; they're not g…

We're probably a long way away from narrative, but dall-e for video is probably only a year or two away from now (they're probably training the model as we speak).

Re: YaLM-100B: Pretrained language model with 100B parameters

#260

Post by brobinson »

londons_explore wrote: post #90

The download fails because the vocab file link returns HTTP 403... :-( https://yalm-100b.s3.yandex.net/vocab/voc_100b.sp EDIT: It seems fine if you download with a browser useragent not CURL... I guess I just got hit by some anti-bot thing they have accidentally have turned on.

curl -A Chrome -O https://yalm-100b.s3.yandex.net/vocab/voc_100b.sp
Post reply on HN