Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

71–80 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#71

Post by redox99 »

Byamarro wrote: post #58
alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

It could be possible with analog chips. I.e. ones that Mythic works on.

I'm not sure why you got downvoted. Yes, ASICs (either analog or digital) that have some model hardcoded in would probably make it feasible, but it won't be programmable which is the interesting part.

Re: YaLM-100B: Pretrained language model with 100B parameters

#72

Post by bloqs »

MrBuddyCasino wrote: post #44
niek_pas wrote: post #36

Earlier quoted context omitted.

Are American developers’ achievement tainted by the fact they come from the United States?

Underrated comment, considering the history of NATO expansion, color revolutions and the hundreds of thousands killed in pursuit of reckless ideological overt or covert warfare.

Yikes

Re: YaLM-100B: Pretrained language model with 100B parameters

#73

Post by redox99 »

albertzeyer wrote: post #32
schizo89 wrote: post #2

I hope one day it will be possible to run this kind of models at home.

If your disk has enough space to store the model, I think in theory you could run them, using the disk to store states. But it will be slow. I'm not sure how slow though, and also if anyone has implemented this. It actually should not be too difficult.

Disk makes no sense considering RAM is pretty cheap. But even then RAM is way too slow (and the communication overhead way too high). You probably get like a 100x slowdown or more.

Re: YaLM-100B: Pretrained language model with 100B parameters

#74

Post by H8crilA »

alexb_ wrote: post #60
espadrine wrote: post #50

Earlier quoted context omitted.

> I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Isn’t that already the case? Sure, it costs $60K, but that is accessible to a surprisingly large minority, considering the potency of this software.

...what? 60 thousand dollars for a dedicated computer that you can't use is not everyone, not on their own computers, and is also a crazy large amount of money for nearly everyone. Sure there are some that could, but that's not what I said.

Indeed. What "everyone" can use is a ~$200 smartphone, so there's a ~300x gap to be bridged.

Re: YaLM-100B: Pretrained language model with 100B parameters

#75

Post by wongarsu »

jstimpfle wrote: post #42
hnechochamber2 wrote: post #17

Earlier quoted context omitted.

$ dd if=/dev/zero of=/swapfile bs=1G count=250 status=progress $ chmod 600 /swapfile $ mkswap -U clear /swapfile $ swapon /swapfile

Is there a reason why it is required to fill the swapfile with zeroes here? Normally you'd see something like "dd of=/swapfile bs=1G seek=3 count=0", creating a file of size 3G but with no space allocated (yet). It's much quicker to complete the setup this way.

I assume if you force the file system to allocate inodes you are likely to have a less fragmented file than if you create a sparse file that gets inodes assigned over time when each part is used.

Re: YaLM-100B: Pretrained language model with 100B parameters

#77

Post by arathore »

alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

If by running models you mean just the inference phase, then even today you can run large family of ML models on commodity hardware (with some elbow grease, of course). The training phase is generally the one not easily replicated by non-corporations.

Re: YaLM-100B: Pretrained language model with 100B parameters

#79

Post by layer8 »

pastacacioepepe wrote: post #29
baxtr wrote: post #23

Earlier quoted context omitted.

"We want to punish you" and "We don't want to make business with you" are two very different things IMHO.

If you go out of your way to not do business with someone, in order to cripple them economically, then it is a punishment.

The purpose is deterrence, not punishment.

Re: YaLM-100B: Pretrained language model with 100B parameters

#80

Post by Comevius »

alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

That's definitely the future, personalized entertainment and social interactions will be big. I could watch a movie made for me, and discuss it with a bunch of chat bots. The future will be bubbly as hell, people will be decaying in their safe places as the hellscape rages on outside.
Post reply on HN