Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

441–450 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#441

Post by Svoka »

danuker wrote: post #9
egorfine wrote: post #6

I have huge respect for developers at Yandex. It's kind of sad that achievements like these are tainted by the fact that they come from Russia (and I speak as a Ukrainian). I wonder if the permissive license is able to mitigate that.

Coming from Russia doesn't mean you agree with government policy. If you saw people get arrested as soon as they start protesting, what would you do?

Every russian citizen pays for death and destruction in Ukraine. With taxes, with national wealth.

What should russians do you ask? Fight. I did it in Ukraine in 2004. Then in 2014. I didn't run from cops, I didn't let them take my friends. But regardless, now we pay with our lives, being subjected to genocide because of russian cowardliness.

because so far they are all just paying for the genocide.

Re: YaLM-100B: Pretrained language model with 100B parameters

#442

Post by ChuckNorris89 »

riku_iki wrote: post #353
waffleiron wrote: post #341

Earlier quoted context omitted.

Weird how there is limited hard evidence of a secret, illegal government program... It's a lot more than I've seen than evidence for the claims of Yandex proactively sharing data with the Russian government. Also where do you see Venezuela?

So, no proof, no evidence. Ok. > It's a lot more than I've seen than evidence for the claims of Yandex proactively sharing data with the Russian government. The difference is that checks and balances are much stronger in US, and such activities can be successfully investigated and government sued. As an example, your verizon case was successfully challenged: https://en.wikipedia.org/wiki/Klayman_v._Obama In Russia, c…

>So, no proof, no evidence.

Do you really expect the US government to literally publish their illegal surveillance operations on Wikipedia as proof?

Snowden's leaks and his statements should be enough to understand the big-tech surveillance apparatus aids the government under the table.

Re: YaLM-100B: Pretrained language model with 100B parameters

#443

Post by trasz »

5e92cb50239222b wrote: post #236
trasz wrote: post #189

Earlier quoted context omitted.

Yeah, it's certainly about ethnicity, not at all about being controlled by a government which is in the process of perpetrating genocide.

I'm ethnically Russian (mostly), although I've never been to that country and have less influence on their foreign policy than your average European (who at least has some say in how his own country behaves towards Russia — and we've seen how well they managed that). I don't know how this would translate to the real world if I lived in "the West", but from what I'm seeing on the internet for the past few months, it d…

How is your Russian ethnicity different from average Ukrainian?

But yes, there is, sadly, some discrimination. It's got nothing to do with ethnicity though; it's the same thing that happened to ordinary Germans in 1939-1945, and for the same reasons.

Re: YaLM-100B: Pretrained language model with 100B parameters

#444

Post by time_to_smile »

alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

Comments like this make me feel like I'm losing my mind.

I think it's far more likely that in 10 years we'll all become more used to rolling blackouts, and fondly remember we all used to be able to afford to eat out, and laugh over a glass of cheap gin about how wild things were back in the old days before things got really bad.

10 years ago was a much more exciting and hopeful time than today. I remember watching Hinton show off what deep learning was just starting to do. It was frankly more interesting that high parameter language models. Startups were all working on some cool problems rather than just trying to screw over customers.

That's just technology. Economically, socially and ecologically things looks far brighter in 2012 than they do now, and in 2032 I suspect we'll feel the same about today, but far more dramatically.

We've already pass the peak of "things are getting better all the time!" but people are just in denial about this.

Re: YaLM-100B: Pretrained language model with 100B parameters

#445

Post by iforgotpassword »

dav_Oz wrote: post #150
alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

The bots/machine vs human reminds me of that famous experiment from the 30s in which Winthrop Kellogg[0], a comparative psychologist, and his wife decided to raise their human baby (Donald) simultaneously with a chimpanzee baby (Gua) in an effort to "humanize the ape". It was set out to last 5 years but was relatively quickly abrupted after only 9 months. The explicit reason wasn't stated only that it successfully pr…

It's the commonly believed reason; the child starting to take on habits from Gua, like noises when she wanted something, and the way monkeys scratch themselves. No authoritative source for it though, it's what I've been told during a lecture back in college, and I think PlainlyDifficult mentions it too in their video about it.

https://youtu.be/VP8DD9TGNlU

Re: YaLM-100B: Pretrained language model with 100B parameters

#446

Post by rexpop »

freewizard wrote: post #285
dav_Oz wrote: post #150

Earlier quoted context omitted.

The bots/machine vs human reminds me of that famous experiment from the 30s in which Winthrop Kellogg[0], a comparative psychologist, and his wife decided to raise their human baby (Donald) simultaneously with a chimpanzee baby (Gua) in an effort to "humanize the ape". It was set out to last 5 years but was relatively quickly abrupted after only 9 months. The explicit reason wasn't stated only that it successfully pr…

So maybe the Turing Test is not about AI are smart enough, but about how stupid humans become?

Not stupid; imaginative and agreeable.

Re: YaLM-100B: Pretrained language model with 100B parameters

#447

Post by d23 »

afroboy wrote: post #265
bgandrew wrote: post #155

Earlier quoted context omitted.

no it's not. they straight up serve kremlin, promoting kremlin fake news and silencing russian opposition (not much to silence but still). they can have whatever functionality they like, I still won't use it in billion years.

Literally what google doing in favor of USA.

I would assume this could go unsaid, but apparently it needs to be said somewhere in this thread: there is zero comparison between the US and an autocratic dictator who attempts to kill and then jails his opposition, runs fraudulent elections, kills journalists, and invades sovereign countries. Zero. None. Zero.

Zero.

Get it?

None.

Zero.

Re: YaLM-100B: Pretrained language model with 100B parameters

#448

Post by whimsicalism »

rllearneratwork wrote: post #432
ma2rten wrote: post #381

I am one of the people who worked on Google's PaLM model. Having skimmed the GitHub readme and medium article, this announcement seems to be very focused on the number of parameters and engineering challenges scaling the model, but it does not contain any details about the model, training (learning rate schedules, etc.), or data composition. It is great that more models are getting released publicly, but I would not…

Given that Yandex is a crucial part of Russian propaganda arm, we should consider the whole range of possibilities from: * Good. This is great researchers helping community by sharing great work. (which is what I'd like to assume before I have any proof of the contrary) * Bad. This very expensive training has been approved by Ya leadership (which is under Western personal sanctions) because they've secretly built in…

Should we assume language models released by Twitter have injected content praising Hunter Biden?

Re: YaLM-100B: Pretrained language model with 100B parameters

#449

Post by rllearneratwork »

whimsicalism wrote: post #448
rllearneratwork wrote: post #432

Earlier quoted context omitted.

Given that Yandex is a crucial part of Russian propaganda arm, we should consider the whole range of possibilities from: * Good. This is great researchers helping community by sharing great work. (which is what I'd like to assume before I have any proof of the contrary) * Bad. This very expensive training has been approved by Ya leadership (which is under Western personal sanctions) because they've secretly built in…

Should we assume language models released by Twitter have injected content praising Hunter Biden?

No. read my message again. As I said, we should assume good intention first until proven otherwise.

But we should have better tools to test for biases/toxicity. Perspective API is great tool for toxicity detection. But I'm not aware of any "propoganda" detection tool.

Re: YaLM-100B: Pretrained language model with 100B parameters

#450

Post by thatwasunusual »

Planktonne wrote: post #185
Comevius wrote: post #80

Earlier quoted context omitted.

That's definitely the future, personalized entertainment and social interactions will be big. I could watch a movie made for me, and discuss it with a bunch of chat bots. The future will be bubbly as hell, people will be decaying in their safe places as the hellscape rages on outside.

> I could watch a movie made for me We're a long, long way from this. Stringing words/images together into a coherent sequence is arguably the easy bit of creating novels/films, and computers still lag a long way behind humans in this regard. Structuring a narrative is a harder, subtler step. Our most advanced ML solutions are improving rapidly, but often struggle with coherence over a single paragraph; they're not g…

> Structuring a narrative is a harder, subtler step.

You can say that about many movies/series made entirely by humans today. :)

Post reply on HN