Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

521–530 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#521

Post by importantbrian »

Planktonne wrote: post #185
Comevius wrote: post #80

Earlier quoted context omitted.

That's definitely the future, personalized entertainment and social interactions will be big. I could watch a movie made for me, and discuss it with a bunch of chat bots. The future will be bubbly as hell, people will be decaying in their safe places as the hellscape rages on outside.

> I could watch a movie made for me We're a long, long way from this. Stringing words/images together into a coherent sequence is arguably the easy bit of creating novels/films, and computers still lag a long way behind humans in this regard. Structuring a narrative is a harder, subtler step. Our most advanced ML solutions are improving rapidly, but often struggle with coherence over a single paragraph; they're not g…

I actually wouldn't be surprised if the technology catches up to this faster than we realize. I think the actual barrier to large scale adoption of it will be financial and social incentives.

A big reason all the major studios are moving to big franchises is that the real money is in licensing the merch. The movies and TV shows are really just there to sell more merch. Maybe this will work when we all have high quality 3d printers at our desks and we can just print the merch they sell us.

The other big barrier is social. A lot of what people watch, they watch because it was recommended to them by friends or colleagues, and they want to talk about what other people are talking about. I'm sure that there will be many people who will get really into watching custom movies and discussing those movies with chatbots, but I bet most people will still want to socialize and discuss the movies they watch with other humans. FOMO is an underestimated driver of media consumption.

Re: YaLM-100B: Pretrained language model with 100B parameters

#522

Post by hansonw »

idealmedtech wrote: post #291

In the download script, it skips parts of the model (02 and 83); any ML people have ideas why you'd do that?

It appears the indexing for the model parts is deliberately not contiguous; the 03-82 range represents the main 80 transformer layers. https://github.com/yandex/YaLM-100B/blob/main/megatron_lm/me...

Re: YaLM-100B: Pretrained language model with 100B parameters

#523

Post by spaniard89277 »

azinman2 wrote: post #418
colordrops wrote: post #363

Earlier quoted context omitted.

You're going to get downvoted, but Eric Schmidt worked regularly with the state department, and google employees were involved in spurring the color revolutions. Julian Assange detailed this in a newsweek article before his name and body were smeared into the ground: https://www.newsweek.com/assange-google-not-what-it-seems-27... Oh, but they say he's not trustworthy, or that it's a conspiracy theory that he was inte…

I think you ran out of tinfoil this one is so large.

Yeah first thing we hear of US gov using tech companies for spionage and data mining. So much tinfoil yadda yadda

Re: YaLM-100B: Pretrained language model with 100B parameters

#524

Post by lukestateson »

1. Yandex supports the Russian Terrorist regime.

2. Yandex News service ignores the genocide currently happening in Ukraine.

3. Yandex Search engine hides the pictures of Bucha and Irpin massacre as well as Kharkiv and Mariupol destruction.

Yandex using whitewashing tactics via open source.

Re: YaLM-100B: Pretrained language model with 100B parameters

#526

Post by gambler »

lostmsu wrote: post #479
gambler wrote: post #474

Earlier quoted context omitted.

https://medium.com/dan-sanchez/don-t-see-evil-148ae18bc9fe https://citizenactionmonitor.wordpress.com/2017/08/02/google... https://www.protocol.com/bulletins/google-censors-war

Still no. First case I would not even consider censorship. The third one was temporary until Google stopped operating in Russia altogether. A quote from the second one: "cumulative 45 percent decrease in traffic from Google searches"

There is a difference between "Google does not censor anti-war content" and "Google does censor anti-war content, but usually has an excuse I find acceptable".

When a company puts Jon Lennon's Merry Xmas (War is Over) behind age restriction banner[1], the question stops being "Is there censorship?" and becomes about the logic of such censorship.

>The third one was temporary until Google stopped operating in Russia altogether.

They've censored other things on behest of the Russian government for years[2]. Again, I cannot fathom how people on a tech website like HN can be unaware of such things. This is common knowledge broadly covered on mainstream websites.

---

[1] https://reclaimthenet.org/youtube-john-lennon-war-is-over-wa...

[2] https://www.rferl.org/a/google-censors-search-results-after-...

Re: YaLM-100B: Pretrained language model with 100B parameters

#528

Post by vgel »

lumost wrote: post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

My guess is they're mostly vanity projects for large tech companies. While the models have some value, they also serve as interesting research projects and help them attract ML talent to work on more profitable models like ad-targeting.

Re: YaLM-100B: Pretrained language model with 100B parameters

#529

Post by kfrzcode »

justinlloyd wrote: post #498
qayxc wrote: post #467

Earlier quoted context omitted.

200+ GiB of RAM still sounds like a pretty steep hardware requirement.

Oh yeah, that $750 for 256GB of DDR-4 is going to totally break the bank.

Damn I didn't know ram was so cheap

Re: YaLM-100B: Pretrained language model with 100B parameters

#530

Post by PartiallyTyped »

thejosh wrote: post #65
ketzu wrote: post #21

Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.

It's also a power issue. The 4090 sounds like you're going to need a much, MUCH higher PSU than you currently use.. or it'll suddenly turn off as it uses 2-3x the power. You'll need your own wiring to run your PC soon :-)

I bought a 1500w psu soon after the previous crypto collapse for around $150, one of the best purchases I did.
Post reply on HN