Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

551–560 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#551

I love Yandex. They are the best search engine by far for politically controversial topics. They also release a language model to benefit everyone even if it says politically incorrect stuff. They also name their projects "cocaine" probably to perhaps to prevent western competitors from using them. You look at OpenAI and how they don't release their models mainly because they fear "bad people" will use them for "bad…

Is this sarcasm?

Re: YaLM-100B: Pretrained language model with 100B parameters

#552
post #545

Earlier quoted context omitted.

> I suppose they can't just shut it down because the government threatens to nationalize Yandex in response They can destroy equipment, safely delete all the code repositories etc. beforehand, thus rendering the company useless before the nationalization. But $$$ is more important. > Can you please show any public statement from Yandex from which one could derive that? Yandex pays tens/hundreds of millions in taxes a…

> Yandex pays tens/hundreds of millions in taxes and thus finances the war. So what? You shut down the business with 20k employees, on the grounds that you do not agree with local regulations or because the government did bad? That is as far from reality as it gets. > But $$$ is more important Yeah, I think preserving the company is more important than that proposed suicide move (that wouldn't have worked anyway beca…

> It's not just money, it's people, it's culture, it's all the great projects the company does.

What about people killed by the Russian army, sponsored by Yandex?

I guess those matter less than the company culture, right?

Re: YaLM-100B: Pretrained language model with 100B parameters

#553
post #550

I love Yandex. They are the best search engine by far for politically controversial topics. They also release a language model to benefit everyone even if it says politically incorrect stuff. They also name their projects "cocaine" probably to perhaps to prevent western competitors from using them. You look at OpenAI and how they don't release their models mainly because they fear "bad people" will use them for "bad…

>They are the best search engine by far for politically controversial topics. This is an interesting take given the political censorship in Russia (for some ineffable reason much harsher now than it used to be 4 months ago) and cases like https://twitter.com/kevinrothrock/status/1510944781492531208 .

Search Google and Yandex for "2020 election fraud." The results are VERY different. The Zach Vorhies leak shows that Google regularly does blatant censorship for political purposes.[1]

[1]https://www.breitbart.com/tech/2021/08/19/google-whistleblow...

Re: YaLM-100B: Pretrained language model with 100B parameters

#554
post #2

I hope one day it will be possible to run this kind of models at home.

I think that unlikely. Barring some breakthrough that takes us beyond the limits of silicon.

Couldn’t the same thing be said about most things we do on our phones these days?

Won’t incremental advancement cover this eventually? (i.e. no major breakthrough required, just patience).

Re: YaLM-100B: Pretrained language model with 100B parameters

#555
post #537
post #534

Earlier quoted context omitted.

> That is actually required by the Russian regulations on news aggregator services. I Was Just Following Orders (c) Yandex could just shut down Yandex.News service completely years ago without repercussions. They choose not to.

> without repercussions That comes from where? The repercussions could have been very severe. The Russian government easily takes over and seizes control over "rogue companies". Russia is not a free country, my friend.

I am from Russia (though moved to Turkiye once war began) and I do have several friends working at Yandex on different positions including some quite high in management. So I well aware about their reasoning behind keeping working at Yandex.

Basically even today after war has began and tens of thousands were killed on both sides some of people working there still hold the illusion that they could continue to live in their bubble and continue to innovate in Russia like nothing happen. So no, they are not some poor IT company opressed by the government. Every employee who wanted to immigrate was able to move abroad.

6-10 years ago Yandex can certainly shut down their news service without being seized. Back in 2008-2012 one of Yandex co-founders and ex-CTO Ilya Segalovich was often visitor of street protests almost until his death in 2013 and this did not caused company to be seized.

Re: YaLM-100B: Pretrained language model with 100B parameters

#556
post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

There are tons of commercial uses for these models. I've been experimenting with an app targeted toward language learners [1]. We use large language models to:

- Generate vocabulary - e.g. for biking: handlebars, pedals, shifters, etc

- Generate translation exercises for given topic a learner wants to learn about - e.g. I raised the seat on my bike

- Generate questions for the user - e.g. What are the different types of biking?

- Provide more fluent ways to say things - I went on my bike to the store -> I rode my bike to the store

- Provide explanations of the difference in meaning between two words

And we have fine tuned smaller models to do other thing like grammar correction, exercise grading, and embedded search.

These models are going to completely change the field of education in my opinion.

1) https://squidgies.app - be kind it's still a bit alpha

Re: YaLM-100B: Pretrained language model with 100B parameters

#557
post #550

Earlier quoted context omitted.

>They are the best search engine by far for politically controversial topics. This is an interesting take given the political censorship in Russia (for some ineffable reason much harsher now than it used to be 4 months ago) and cases like https://twitter.com/kevinrothrock/status/1510944781492531208 .

Search Google and Yandex for "2020 election fraud." The results are VERY different. The Zach Vorhies leak shows that Google regularly does blatant censorship for political purposes.[1] [1] https://www.breitbart.com/tech/2021/08/19/google-whistleblow...

I don't know man, "thegatewaypundit.com" as a top reputable source? seems to me like it's not "honest two-sided results" but just, well, a rather random mix of result of widely varying quality. Mad Altavista vibes!

What I'm trying to say is that even if you believe that "was the 2020 US election stolen?" is worth debating, which it isn't, the yandex results are shit.

Re: YaLM-100B: Pretrained language model with 100B parameters

#558
post #550

Earlier quoted context omitted.

>They are the best search engine by far for politically controversial topics. This is an interesting take given the political censorship in Russia (for some ineffable reason much harsher now than it used to be 4 months ago) and cases like https://twitter.com/kevinrothrock/status/1510944781492531208 .

Search Google and Yandex for "2020 election fraud." The results are VERY different. The Zach Vorhies leak shows that Google regularly does blatant censorship for political purposes.[1] [1] https://www.breitbart.com/tech/2021/08/19/google-whistleblow...

Google: 118M results. Top link is the best resource on verified election fraud cases.

Yandex: 9M results. The top two links are pretty suspect. Top link promotes Dinesh D'Souza's 2000 Mules documentary in the banner which at best is a one-sided take on election fraud. At worst, very misleading.

https://i.imgur.com/n5a9LOd.png

Re: YaLM-100B: Pretrained language model with 100B parameters

#559
post #505

Earlier quoted context omitted.

From the readme, it looks like you need that RAM on your GPU.

There isn't any reason you can't run a neural net on a CPU. It's still just a bunch of big matrix operations. The advantage of the GPU is it's a lot faster, but "a lot" might be 1 second versus 10 seconds, and for some applications 10 seconds of inference latency is just fine (I have no idea how long this model would take). All the major ML libraries will operate in CPU-only mode if you request it.

They are pretty slow even on GPU. The problem is that it's an autoregressive model. So it needs to do a forward pass for each token.

Re: YaLM-100B: Pretrained language model with 100B parameters

#560
post #545

Earlier quoted context omitted.

> Yandex pays tens/hundreds of millions in taxes and thus finances the war. So what? You shut down the business with 20k employees, on the grounds that you do not agree with local regulations or because the government did bad? That is as far from reality as it gets. > But $$$ is more important Yeah, I think preserving the company is more important than that proposed suicide move (that wouldn't have worked anyway beca…

> It's not just money, it's people, it's culture, it's all the great projects the company does. What about people killed by the Russian army, sponsored by Yandex? I guess those matter less than the company culture, right?

The Russian army is not sponsored by Yandex. The money comes from selling natural resources... and mostly to Europe, surprise. It's about $1 billion per day. Tax money from private companies is nothing compared to that. So Europe is sponsoring the war way more than Yandex.

Let's then shut down the Europe, right? You can say — look, they're trying hard to get rid of Russian resources. But Yandex is also trying hard to become less dependent on Russian economy — they try to internationalize their business. And all that "canceling" of Yandex really doesn't help (it does the opposite in fact).

Post reply on HN