Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

261–270 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#261

Post by sanxiyn »

SXX wrote: post #249
daniel-cussen wrote: post #243

Earlier quoted context omitted.

You know 100 years ago you could just buy uranium openly? Leo Szílard hustled up 200 kilograms, pleted, in the 30's.

What does it have to do with OpenAI branding? Their "moral" reasoning behind not publishing models is simply laughtable because they do sell API access to them to anyone who can pay. And "bad guys" generally have money.

They can (and do) revoke API access from bad guys. They can't do that to downloaded models. Look, I don't like what OpenAI does, but "API access, but no model download" makes sense if you are worried about misuses.

Re: YaLM-100B: Pretrained language model with 100B parameters

#262

Post by sailingparrot »

ketzu wrote: post #21

Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.

If you don't care about inference speed being in the 1-5sec range, then that should be doable with CPU offloading, with e.g. DeepSpeed.

Re: YaLM-100B: Pretrained language model with 100B parameters

#264

Post by Havoc »

MrBuddyCasino wrote: post #53
ketzu wrote: post #21

Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.

Wondering if Apple Silicon will bring arge amounts of unified main memory with high bandwidth to the masses? The Mac Studio maxes out at 128GB currently for around $5K, so 256GB isn't that far out and might work with the ~200GB Yandex says is required.

Perhaps on quantity. Substantially slower though around ~3x from what I can tell…substantial roadblock if you’re training models that take weeks.

Re: YaLM-100B: Pretrained language model with 100B parameters

#265

Post by afroboy »

bgandrew wrote: post #155
kome wrote: post #139

I agree, yandex is a great search engine

no it's not. they straight up serve kremlin, promoting kremlin fake news and silencing russian opposition (not much to silence but still). they can have whatever functionality they like, I still won't use it in billion years.

Literally what google doing in favor of USA.

Re: YaLM-100B: Pretrained language model with 100B parameters

#267

Post by yonixw »

OmicronCeti wrote: post #226
yonixw wrote: post #173

Earlier quoted context omitted.

Bad comment. First, can you please name the founder? Because according to wiki Ilya Segalovich never lived in Israel and Arkady_Volozh lives in Tel Aviv (Not a settlement). Both Jewish, so why present it as some "must-be-hidden-cause" connection with Israel? Also, nothing shady from Israel side in term of sanctions. They have a large Jewish community in both Russia and Ukraine and need to be on good term with both to…

>Elena Bunina, who is Jewish, is stepping down from her role as CEO of Yandex LLC, 'Russia's Google,' amid the war in Ukraine. Sources confirm she is in Israel and has no intention of returning to Russia https://www.haaretz.com/israel-news/tech-news/2022-04-06/ty-...

It doesn't say she lives in a settlement. Or have any special treatment from Israel or influence on Israel in term of sanctions.

Re: YaLM-100B: Pretrained language model with 100B parameters

#268

Post by blackhaz »

ushakov wrote: post #198
bgandrew wrote: post #155

Earlier quoted context omitted.

no it's not. they straight up serve kremlin, promoting kremlin fake news and silencing russian opposition (not much to silence but still). they can have whatever functionality they like, I still won't use it in billion years.

can you name any Russian company that doesn't? obeying to Kremlin is just an aspect of running business in Russia the only option would be not to operate in Russia at all. Yandex can't do this, because their audience is primarily in Russia

Which doesn't excuse them at all. They are full-on supporting the war machine and bloodshed and bear responsibility.

Re: YaLM-100B: Pretrained language model with 100B parameters

#269

Post by jeroenhd »

denysvitali wrote: post #102
lukestateson wrote: post #99

[flagged]

I don't want to defend them, but I'm genuinely curious: aren't they maybe doing it because the opposite will cause them huge legal issues?

This is often overlooked and it's a fair point in defence of the people working for Yandex. You can't judge someone just for working for Yandex or even most Russian companies. The people who have voiced concern are already out of the company and it's perfectly reasonable that the rest would like to keep their jobs, especially in uncertain economic times with all these sanctions against Russia.

However, this also implies that Yandex, as a company, cannot be trusted. It's not the researcher's fault, but they simply aren't allowed to work in a way that doesn't reinforced the Russian government's bias. As usual, the Russian government is the real villain here, but its authoritarian rule "infects" any company and country it has control over.

It can be assumed that the people working for Yandex are also victims of their abusive government, but that doesn't change the fact that their work is unlikely to be trusted outside the Russian sphere of influence.

Re: YaLM-100B: Pretrained language model with 100B parameters

#270

Post by [deleted] »

Planktonne wrote: post #185
Comevius wrote: post #80

Earlier quoted context omitted.

That's definitely the future, personalized entertainment and social interactions will be big. I could watch a movie made for me, and discuss it with a bunch of chat bots. The future will be bubbly as hell, people will be decaying in their safe places as the hellscape rages on outside.

> I could watch a movie made for me We're a long, long way from this. Stringing words/images together into a coherent sequence is arguably the easy bit of creating novels/films, and computers still lag a long way behind humans in this regard. Structuring a narrative is a harder, subtler step. Our most advanced ML solutions are improving rapidly, but often struggle with coherence over a single paragraph; they're not g…

[deleted]
Post reply on HN