Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

601–610 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#601

Post by dang »

lukestateson wrote: post #594
dang wrote: post #543

Earlier quoted context omitted.

You've posted 7 highly repetitive comments taking this thread straight into flamewar hell. That's not what this site is for, and destroys what it is for. If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules, we'd appreciate it. Hijacking top comments when flamebait hasn't succeeded in setting an entire thread on fire yet is particularly abusive. We detached this subthread fro…

You think I care about your comment? I’m being bombed by Russian terrorists each day. Lol all I care about is to just live another day.

That's awful and I can't imagine how difficult it is, and I certainly hope that you come out of it ok.

Re: YaLM-100B: Pretrained language model with 100B parameters

#602

Post by can16358p »

HeavyStorm wrote: post #492
lumost wrote: post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

Maybe training it is not that expensive? I know from practice that it takes a really really long time to train even a small nn (thousands of params) , so you'll need a lot more hardware to train one with billions... But, it's expensive to buy the hardware, not necessarily to use it. If you, for some reason, have a few hundred GPU lying around, it might be "cheap" to do the necessary training. Now, that's not your poi…

I can't think of anyone having a few hundred GPUs around unless:

- They were into Ethereum mining and quit.

- They've already built a cluster with them (e.g. in an academic setting).

- They live in a datacenter.

- They are a total psychopath.

But even assuming one magically has all those GPUs available and ready to train, I don't want to calculate the power cost of it anyway. Unless one has access to free or extremely cheap electricity it would still be very expensive.

Re: YaLM-100B: Pretrained language model with 100B parameters

#603

Post by htrp »

natly wrote: post #259
Planktonne wrote: post #185

Earlier quoted context omitted.

> I could watch a movie made for me We're a long, long way from this. Stringing words/images together into a coherent sequence is arguably the easy bit of creating novels/films, and computers still lag a long way behind humans in this regard. Structuring a narrative is a harder, subtler step. Our most advanced ML solutions are improving rapidly, but often struggle with coherence over a single paragraph; they're not g…

We're probably a long way away from narrative, but dall-e for video is probably only a year or two away from now (they're probably training the model as we speak).

https://video-diffusion.github.io/

Google's got you!

Re: YaLM-100B: Pretrained language model with 100B parameters

#604

Post by SR2Z »

narrator wrote: post #577
zaptrem wrote: post #573

Earlier quoted context omitted.

Why link to Breitbart of all places instead of the original source? https://www.cjr.org/tow_center/google-news-algorithm.php Btw Wikipedia’s first few sentences on Breitbart are not inspiring > Its journalists are widely considered to be ideologically driven, and much of its content has been called misogynistic, xenophobic, and racist by liberals and traditional conservatives alike.[10] The site has published a numbe…

This is the association fallacy, which is, unfortunately, how most people determine what to believe these days. An absurd example of this fallacy would be, Wikipedia, which you cite, has articles that indicate tobacco smoking may cause disease. The nazis were also anti-smoking[1]. Therefore Wikipedia is Nazi propaganda and you should not trust anything on there. [1] https://www.amazon.com/Nazi-War-Cancer-Robert-Proct…

It is not the association fallacy; the role of a news site is to provide news, which includes fact-checking the work of their "journalists."

If Breitbart pulled a Fox News and argued in court that their goal was to entertain and not inform, then you have a point! But until then, you have a terrible misunderstanding of journalistic integrity and what it means for a publisher to attach their name to a journalist's work.

Re: YaLM-100B: Pretrained language model with 100B parameters

#605

Post by orbital-decay »

cpursley wrote: post #580
risyachka wrote: post #571

Earlier quoted context omitted.

>> They are the best search engine by far for politically controversial topics FYI, they are Russian subject that follows ALL their censorship laws (and oh boy do they have a lot of it). >> probably to perhaps to prevent western competitors from using them The irony here. All yandex products are exact copies of western, adjusted to local market.

Actually they're not, some of the Yandex products are actually better and pretty innovative (ignoring the political stuff). Maps and Go are especially good. Ditto with Russian banking apps, they put American bank apps to shame.

>some of the Yandex products are actually better and pretty innovative (ignoring the political stuff). Maps and Go are especially good.

Yeah, the same Yandex Maps that stopped showing state borders recently, as they are now "more focused on natural objects", in their words.

Re: YaLM-100B: Pretrained language model with 100B parameters

#606

Post by throwaway_1928 »

narrator wrote: post #508

I love Yandex. They are the best search engine by far for politically controversial topics. They also release a language model to benefit everyone even if it says politically incorrect stuff. They also name their projects "cocaine" probably to perhaps to prevent western competitors from using them. You look at OpenAI and how they don't release their models mainly because they fear "bad people" will use them for "bad…

> Hey, we are the bad guys you're talking about so who are we keeping this technology from?

Laughed out loud!

Re: YaLM-100B: Pretrained language model with 100B parameters

#607

Post by alluro2 »

time_to_smile wrote: post #444
alexb_ wrote: post #13

I have to wonder if 10 years down the line, everyone will be able to run models like this on their own computers. Have to wonder what the knock-on effects of that will be, especially if the models improve drastically. With so much of our social lives being moved online, if we have the easy ability to create fake lives of fake people one has to wonder what's real and what isn't. Maybe the dead internet theory will rea…

Comments like this make me feel like I'm losing my mind. I think it's far more likely that in 10 years we'll all become more used to rolling blackouts, and fondly remember we all used to be able to afford to eat out, and laugh over a glass of cheap gin about how wild things were back in the old days before things got really bad. 10 years ago was a much more exciting and hopeful time than today. I remember watching Hi…

You're not alone - especially based on observations during the pandemic, it seem we are woefully unaware and unprepared for how fragile the structures supporting our current way of living are, and how easy they would collapse into a much worse state of living conditions when it comes to power and food, let alone luxuries and internet as we know it...

It also seems to me that most people would not be ready to give up more than 10% of their luxuries / way of living up-front in order to protect those structures and would continue to watch funny TikTok videos and post IG photos until the very moment their internet access goes out and doesn't come back.

Re: YaLM-100B: Pretrained language model with 100B parameters

#608

Post by frozencell »

true_religion wrote: post #520
sanxiyn wrote: post #311

Earlier quoted context omitted.

OpenAI published "Lessons Learned on Language Model Safety and Misuse" in March. https://openai.com/blog/language-model-safety-and-misuse/ It also promised "forthcoming publication". Examples of "real cases of misuse encountered in the wild" include "spam promotions for dubious medical products and roleplaying of racist fantasies". Yes, some bad actors can train their own models, but OpenAI can't do much about that e…

It would be better for misuse to be criminalized and taken care of by national governments, rather than leave it to for-profit companies to decide what is or isn't "misuse". Personally, I think using AI to manufacture advertisements on demand is misuse... but will Google agree with me?

National governments are often the criminals themselves, or their partner.

Re: YaLM-100B: Pretrained language model with 100B parameters

#609

Post by calrizien »

uniqueuid wrote: post #101
tehsauce wrote: post #97

Earlier quoted context omitted.

upcoming mac pro will have pretty poor ML performance when compared to even an old nvidia gpu sadly.

Although memory capacity may matter more than speed for inference . As long as you're not training or fine tuning, the mac pro / studio may be just fine . apart from the fact that you can't use any of the many nvidia-specific things; if you're dependent on cuda, nvcuvid, AMP or other things that's a hard no.

What are the current best ML language models to play with on the M1Max?

Re: YaLM-100B: Pretrained language model with 100B parameters

#610

Post by make3 »

narrator wrote: post #508

I love Yandex. They are the best search engine by far for politically controversial topics. They also release a language model to benefit everyone even if it says politically incorrect stuff. They also name their projects "cocaine" probably to perhaps to prevent western competitors from using them. You look at OpenAI and how they don't release their models mainly because they fear "bad people" will use them for "bad…

It's widely accepted that OpenAi doesn't release its models to make money from them, not because they really think they would be harmful
Post reply on HN