I love Yandex. They are the best search engine by far for politically controversial topics. They also release a language model to benefit everyone even if it says politically incorrect stuff. They also name their projects "cocaine" probably to perhaps to prevent western competitors from using them. You look at OpenAI and how they don't release their models mainly because they fear "bad people" will use them for "bad…
YaLM-100B: Pretrained language model with 100B parameters
551–560 of 666 posts
Re: YaLM-100B: Pretrained language model with 100B parameters
#552Earlier quoted context omitted.
> I suppose they can't just shut it down because the government threatens to nationalize Yandex in response They can destroy equipment, safely delete all the code repositories etc. beforehand, thus rendering the company useless before the nationalization. But $$$ is more important. > Can you please show any public statement from Yandex from which one could derive that? Yandex pays tens/hundreds of millions in taxes a…
> Yandex pays tens/hundreds of millions in taxes and thus finances the war. So what? You shut down the business with 20k employees, on the grounds that you do not agree with local regulations or because the government did bad? That is as far from reality as it gets. > But $$$ is more important Yeah, I think preserving the company is more important than that proposed suicide move (that wouldn't have worked anyway beca…
What about people killed by the Russian army, sponsored by Yandex?
I guess those matter less than the company culture, right?
Re: YaLM-100B: Pretrained language model with 100B parameters
#553I love Yandex. They are the best search engine by far for politically controversial topics. They also release a language model to benefit everyone even if it says politically incorrect stuff. They also name their projects "cocaine" probably to perhaps to prevent western competitors from using them. You look at OpenAI and how they don't release their models mainly because they fear "bad people" will use them for "bad…
>They are the best search engine by far for politically controversial topics. This is an interesting take given the political censorship in Russia (for some ineffable reason much harsher now than it used to be 4 months ago) and cases like https://twitter.com/kevinrothrock/status/1510944781492531208 .
[1]https://www.breitbart.com/tech/2021/08/19/google-whistleblow...
Re: YaLM-100B: Pretrained language model with 100B parameters
#554I hope one day it will be possible to run this kind of models at home.
I think that unlikely. Barring some breakthrough that takes us beyond the limits of silicon.
Won’t incremental advancement cover this eventually? (i.e. no major breakthrough required, just patience).
Re: YaLM-100B: Pretrained language model with 100B parameters
#555Earlier quoted context omitted.
> That is actually required by the Russian regulations on news aggregator services. I Was Just Following Orders (c) Yandex could just shut down Yandex.News service completely years ago without repercussions. They choose not to.
> without repercussions That comes from where? The repercussions could have been very severe. The Russian government easily takes over and seizes control over "rogue companies". Russia is not a free country, my friend.
Basically even today after war has began and tens of thousands were killed on both sides some of people working there still hold the illusion that they could continue to live in their bubble and continue to innovate in Russia like nothing happen. So no, they are not some poor IT company opressed by the government. Every employee who wanted to immigrate was able to move abroad.
6-10 years ago Yandex can certainly shut down their news service without being seized. Back in 2008-2012 one of Yandex co-founders and ex-CTO Ilya Segalovich was often visitor of street protests almost until his death in 2013 and this did not caused company to be seized.
Re: YaLM-100B: Pretrained language model with 100B parameters
#556To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.
- Generate vocabulary - e.g. for biking: handlebars, pedals, shifters, etc
- Generate translation exercises for given topic a learner wants to learn about - e.g. I raised the seat on my bike
- Generate questions for the user - e.g. What are the different types of biking?
- Provide more fluent ways to say things - I went on my bike to the store -> I rode my bike to the store
- Provide explanations of the difference in meaning between two words
And we have fine tuned smaller models to do other thing like grammar correction, exercise grading, and embedded search.
These models are going to completely change the field of education in my opinion.
1) https://squidgies.app - be kind it's still a bit alpha
Re: YaLM-100B: Pretrained language model with 100B parameters
#557Earlier quoted context omitted.
>They are the best search engine by far for politically controversial topics. This is an interesting take given the political censorship in Russia (for some ineffable reason much harsher now than it used to be 4 months ago) and cases like https://twitter.com/kevinrothrock/status/1510944781492531208 .
Search Google and Yandex for "2020 election fraud." The results are VERY different. The Zach Vorhies leak shows that Google regularly does blatant censorship for political purposes.[1] [1] https://www.breitbart.com/tech/2021/08/19/google-whistleblow...
What I'm trying to say is that even if you believe that "was the 2020 US election stolen?" is worth debating, which it isn't, the yandex results are shit.
Re: YaLM-100B: Pretrained language model with 100B parameters
#558Earlier quoted context omitted.
>They are the best search engine by far for politically controversial topics. This is an interesting take given the political censorship in Russia (for some ineffable reason much harsher now than it used to be 4 months ago) and cases like https://twitter.com/kevinrothrock/status/1510944781492531208 .
Search Google and Yandex for "2020 election fraud." The results are VERY different. The Zach Vorhies leak shows that Google regularly does blatant censorship for political purposes.[1] [1] https://www.breitbart.com/tech/2021/08/19/google-whistleblow...
Yandex: 9M results. The top two links are pretty suspect. Top link promotes Dinesh D'Souza's 2000 Mules documentary in the banner which at best is a one-sided take on election fraud. At worst, very misleading.
Re: YaLM-100B: Pretrained language model with 100B parameters
#559Earlier quoted context omitted.
From the readme, it looks like you need that RAM on your GPU.
There isn't any reason you can't run a neural net on a CPU. It's still just a bunch of big matrix operations. The advantage of the GPU is it's a lot faster, but "a lot" might be 1 second versus 10 seconds, and for some applications 10 seconds of inference latency is just fine (I have no idea how long this model would take). All the major ML libraries will operate in CPU-only mode if you request it.
Re: YaLM-100B: Pretrained language model with 100B parameters
#560Earlier quoted context omitted.
> Yandex pays tens/hundreds of millions in taxes and thus finances the war. So what? You shut down the business with 20k employees, on the grounds that you do not agree with local regulations or because the government did bad? That is as far from reality as it gets. > But $$$ is more important Yeah, I think preserving the company is more important than that proposed suicide move (that wouldn't have worked anyway beca…
> It's not just money, it's people, it's culture, it's all the great projects the company does. What about people killed by the Russian army, sponsored by Yandex? I guess those matter less than the company culture, right?
Let's then shut down the Europe, right? You can say — look, they're trying hard to get rid of Russian resources. But Yandex is also trying hard to become less dependent on Russian economy — they try to internationalize their business. And all that "canceling" of Yandex really doesn't help (it does the opposite in fact).