Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

61–70 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#61

Post by easytiger »

Cu3PO42 wrote: post #35
easytiger wrote: post #28

Earlier quoted context omitted.

I thought cuda had a unified memory system? Maybe I misunderstood

Unified memory exists, but it's not a magic bullet. If a page is accessed that doesn't reside on device memory (i.e. on the GPU), a memcpy is issued to fetch the page from main RAM. While the programming model is nicer, it doesn't fundamentally change the fact that you need to constantly swap data out to main RAM and while not as bad as loading it from the SSD or HDD, that's still quite slow. Integrated GPUs that use…

Sure. Makes sense. So I guess for discrete GPUs the unified memory stuff provides a universal address space but merely abstracts the copying/streaming of the data.

There does seem to be a zero copy concept as well and I've certainly used direct memory access over pcie before on other proprietary devices.

https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/ind...

Re: YaLM-100B: Pretrained language model with 100B parameters

#62

Post by perryizgr8 »

ketzu wrote: post #21

Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.

Nvidia deliberately keeps their consumer/gamer cards limited in memory. If you have a use for more RAM, they want you to buy their workstation offerings like RTX A6000 which has 48G DDR6 RAM or A100 which has 80G.

Re: YaLM-100B: Pretrained language model with 100B parameters

#63

Post by tremarley »

csee wrote: post #49
drno123 wrote: post #34

Earlier quoted context omitted.

Who controls Google? What did Google do to stop the inasion of Iraq? Will Google take responsibility for silent support od war in Iraq?

Your comparison fails a test of facts. Yandex actively censors any perspective not approved by the Kremlin. Google does not do anything comparable to this.

Google absolutely does the same thing.

Re: YaLM-100B: Pretrained language model with 100B parameters

#64

Post by trhway »

Thorentis wrote: post #19
danuker wrote: post #9

Earlier quoted context omitted.

Coming from Russia doesn't mean you agree with government policy. If you saw people get arrested as soon as they start protesting, what would you do?

Doesn't mean you can't be punished for the actions of the government though. See: Western companies and government pulling out of Russia or issuing sanctions on private individuals. I think it's even worse to say "we don't think you're guilty, but we do think you should be punished."

> Western companies and government pulling out of Russia

you can't really blame the companies for not wanting to be associated in any way with a nazist regime genociding a neighboring country.

And Starbucks or Mercedes pulling out of Russia isn't punishment. It is freedom of association and economic activity. Russians are whining about "punishment" because they have no idea about freedom, and that is them getting a bit of taste of it. They think they can plaster whole country with their swastika - "Z" - in enthusiastic support of the bloody genocide while the whole word shouldn't be able to express its disgust at those happenings.

Especially funny how Russians are cry-baby style whining about supposed violation of their property rights by the West sanctions while Russians have been violating property rights of more than 40 million of Ukrainians (even if we don't consider all the mass killing and raping of civilians that Russians have been doing there). The deep and profound disintegration of any morals in my old country is stunning.

>or issuing sanctions on private individuals.

due to the size of their wealth and the de-facto rules of economic activity in Russia those aren't private wealth of private individuals - they are integral part of that nazist regime, and thus they are guilty too.

And for the sanctioned Russian government officials - that is for example the Roskosmos CEO Rogozin, who is one of the main founders of the Russian Nazi movement "Motherland" (people from which has since taken prominent roles across the Russian government and the ruling Party) and who is one of the most prominent voices around Putin and the Putin's favorite, giving a Nazi salute and the end of his Nazi speech at the Russian Nazi march in Moscow. The specific phrase they all give Nazi salute to is "Glory to Russia!".

https://youtu.be/xkXVVcPWSU8?t=87

Re: YaLM-100B: Pretrained language model with 100B parameters

#65

Post by thejosh »

ketzu wrote: post #21

Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.

It's also a power issue. The 4090 sounds like you're going to need a much, MUCH higher PSU than you currently use.. or it'll suddenly turn off as it uses 2-3x the power.

You'll need your own wiring to run your PC soon :-)

Re: YaLM-100B: Pretrained language model with 100B parameters

#66

Post by nmfisher »

guywhocodes wrote: post #59
amelius wrote: post #52

Earlier quoted context omitted.

It's not really random access. I bet the graph can be pipelined such that you can keep a "horizontal cross-section" of the graph in memory all the time, and you scan through the parameters from top to bottom in the graph.

I wonder if you can't do that LSH trick to turn it into a sparse matrix problem and run it on CPU that way.

That's pretty much what SLIDE [0] does. The driver was achieving performance parity with GPUs for CPU training, but presumably the same could apply to running inference on models too large to load into consumer GPU memory.

https://github.com/RUSH-LAB/SLIDE

Re: YaLM-100B: Pretrained language model with 100B parameters

#67

Post by tiborsaas »

alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

I think there will be a trend where model's size will shrink due to better optimization / compression while hardware specs keep increasing.

You can already see this with Chinchilla:

https://towardsdatascience.com/a-new-ai-trend-chinchilla-70b...

Re: YaLM-100B: Pretrained language model with 100B parameters

#68

Post by natly »

alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

I know it's a sort of exaggerated paranoid thought. But like these things do all come down to scale and some areas of the world definitely could have the amount of compute available to make dall-e level quality full scale videos which we might be consuming right now. It really does make you start to wonder at what point we will rationally be able to have zero trust that not everything we watch online is fabricated.

Re: YaLM-100B: Pretrained language model with 100B parameters

#69

Post by baisq »

tremarley wrote: post #63
csee wrote: post #49

Earlier quoted context omitted.

Your comparison fails a test of facts. Yandex actively censors any perspective not approved by the Kremlin. Google does not do anything comparable to this.

Google absolutely does the same thing.

Not to mention that Yandex does it in Russia because the law forces them to, while Google does it happily just to maintain the political status quo, of which they are a part of.

Re: YaLM-100B: Pretrained language model with 100B parameters

#70

Post by nonrandomstring »

alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

> one has to wonder what's real and what isn't.

And whether it really matters. That's the bigger question.

I think, for most of us, it does matter. But we're not sure why and what a loss of human reality would really mean.

For a few who wholeheartedly embrace it there's some resonance with the psychedelic/60s creed that sees this as some kind of "liberation".

Post reply on HN