Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

591–600 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#591

Post by jhgb »

Barrin92 wrote: post #589
jhgb wrote: post #585

Earlier quoted context omitted.

Wait, so you're saying it's a Russian company breaking Russian laws and getting away with it?

> it's a Russian company breaking Russian laws and getting away with it I don't think you've lived in Russia if you need to ask that question. Breaking the law and getting away with it is a way of life in Russia, that goes for all institutions and social strata

Breaking random laws? Sure. Breaking laws specifically made to enable central government control of independent media? Uh, how do you do that? Have you not noticed for how minor things regarding freedom of expression have been Russian people getting into jail recently?

Re: YaLM-100B: Pretrained language model with 100B parameters

#592

Post by Aspie96 »

SXX wrote: post #249
daniel-cussen wrote: post #243

Earlier quoted context omitted.

You know 100 years ago you could just buy uranium openly? Leo Szílard hustled up 200 kilograms, pleted, in the 30's.

What does it have to do with OpenAI branding? Their "moral" reasoning behind not publishing models is simply laughtable because they do sell API access to them to anyone who can pay. And "bad guys" generally have money.

We all know the real reason.

They used to be a non-profit with a mission, now they are a for-profit with the only mission of money.

Re: YaLM-100B: Pretrained language model with 100B parameters

#593

Post by Aspie96 »

JacobThreeThree wrote: post #376
remram wrote: post #316

Earlier quoted context omitted.

Maybe they should rename to SafeAI, if their concern is controlling access.

Good point. The issue is not the policy per se, it's the fact that their name is not accurate.

I mean, the policy is kind of shit too, given that they used to have the mission of being open.

But yes, the name is not what it should be, given their current ideas.

Re: YaLM-100B: Pretrained language model with 100B parameters

#594

Post by lukestateson »

dang wrote: post #543
lukestateson wrote: post #525

[flagged]

You've posted 7 highly repetitive comments taking this thread straight into flamewar hell. That's not what this site is for, and destroys what it is for. If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules, we'd appreciate it. Hijacking top comments when flamebait hasn't succeeded in setting an entire thread on fire yet is particularly abusive. We detached this subthread fro…

You think I care about your comment? I’m being bombed by Russian terrorists each day. Lol all I care about is to just live another day.

Re: YaLM-100B: Pretrained language model with 100B parameters

#595

Post by px43 »

narrator wrote: post #553
abra0 wrote: post #550

Earlier quoted context omitted.

>They are the best search engine by far for politically controversial topics. This is an interesting take given the political censorship in Russia (for some ineffable reason much harsher now than it used to be 4 months ago) and cases like https://twitter.com/kevinrothrock/status/1510944781492531208 .

Search Google and Yandex for "2020 election fraud." The results are VERY different. The Zach Vorhies leak shows that Google regularly does blatant censorship for political purposes.[1] [1] https://www.breitbart.com/tech/2021/08/19/google-whistleblow...

Totally, just like how if you want to find out what really happened in Tiananmen Square in 1989, your best bet is Baidu. Totally different results than what Google gives you!

I sincerely have deep respect for Yandex for releasing this, and Baidu for some of the amazing research they've released over the years, but both are deeply deeply beholden to their local governments in a way that is incomparable to the relationship between Google and the US government.

Remember that the NSA was literally digging up and tapping fiber around Google data centers in a secret program called MUSCULAR because they didn't think Google was being cooperative enough when handing over data that they were requesting.

https://en.wikipedia.org/wiki/MUSCULAR_(surveillance_program...

Re: YaLM-100B: Pretrained language model with 100B parameters

#596

Post by Barrin92 »

jhgb wrote: post #591
Barrin92 wrote: post #589

Earlier quoted context omitted.

> it's a Russian company breaking Russian laws and getting away with it I don't think you've lived in Russia if you need to ask that question. Breaking the law and getting away with it is a way of life in Russia, that goes for all institutions and social strata

Breaking random laws? Sure. Breaking laws specifically made to enable central government control of independent media? Uh, how do you do that? Have you not noticed for how minor things regarding freedom of expression have been Russian people getting into jail recently?

>Uh, how do you do that?

By being on the internet. Russia has always been good at literally hitting you with a physical club if you're crazy enough to take a sign to the streets, but the Russian state doesn't understand the internet, or really anything that's sort of underground or intangible.

There's a reason the country is probably the world's largest place for all things piracy related, scihub and so on. It's not just laxer IP laws, it's also that tech in particular has always skirted all kinds of regulation freely, it's why the country has a relatively healthy tech industry despite at times suffocating regulation. The prevalence of cybercrime in the country is another example of it. Being censorious doesn't make you competent.

Even Telegram which was at some point supposedly blocked was still used by everyone, including funnily enough the foreign ministry itself. These things never really work in Russia.(https://www.reuters.com/article/us-russia-telegram-ban-idUSK...)

Re: YaLM-100B: Pretrained language model with 100B parameters

#597

Post by braingenious »

braingenious wrote: post #576

This is one of the funniest threads I’ve ever seen on this website. People are yelling at eachother about the CIA and the legitimacy of Israel and Assange and the definition of fascism and… anything that pisses anybody off about international politics in general. In a thread about a piece of software that’s (to me and likely many others) prohibitively expensive to play around with. Anyway I hope somebody creates a pl…

What if Street Sharks were mormon missionaries? How would Emily Dickinson describe Angie Dickinson in a poem? How would Ramses II have used Bitcoin?

THESE are the important things to talk about when it comes to this topic.

Re: YaLM-100B: Pretrained language model with 100B parameters

#598

Post by 1024core »

stainablesteel wrote: post #590
alphabetting wrote: post #558

Earlier quoted context omitted.

Google: 118M results. Top link is the best resource on verified election fraud cases. Yandex: 9M results. The top two links are pretty suspect. Top link promotes Dinesh D'Souza's 2000 Mules documentary in the banner which at best is a one-sided take on election fraud. At worst, very misleading. https://i.imgur.com/n5a9LOd.png

This is a weird comment because yes, that's exactly what the above person was saying. It shows results that google won't give you. Secondly, I've yet to see any criticisms of 2000 mules data that aren't addressed by the stringency in the analysis they claim to have done. I thought the information they presented was extremely valuable. Are we going to overturn an election at this point? No. But the vulnerabilities to…

> then clearly taken advantage of.

If you have any evidence, any evidence at all, of significant mail-in ballots fraud, then you should write it up and publish it; and even present it to the USDOJ, because you would have succeeded where Trump's highly-paid teams of lawyers failed.

If you don't have proof, then please STFU.

Re: YaLM-100B: Pretrained language model with 100B parameters

#599

Post by fswd »

ma2rten wrote: post #381

I am one of the people who worked on Google's PaLM model. Having skimmed the GitHub readme and medium article, this announcement seems to be very focused on the number of parameters and engineering challenges scaling the model, but it does not contain any details about the model, training (learning rate schedules, etc.), or data composition. It is great that more models are getting released publicly, but I would not…

it's in there look for this sentence. And they did some top dog stuff: Training details and best practices on acceleration and stabilizations can be found on Medium (English)

Re: YaLM-100B: Pretrained language model with 100B parameters

#600

Post by TaylorAlexander »

rahidz wrote: post #579
MasterScrat wrote: post #570

Earlier quoted context omitted.

HuggingFace will soon release their BigScience model: https://twitter.com/BigScienceLLM/status/1539941348656168961 "a 176 billion parameter transformer model that will be trained on roughly 300 billion words in 46 languages" So anything smaller than that will become worthless. May be a factor, companies have a last chance to make a PR splash before it happens. Read more about it: https://bigscience.huggingface.co/blo…

Not necessarily, only ~30% of the database is in English, so it likely won't be as good as a smaller model trained solely or mostly on English words. https://bigscience.huggingface.co/blog/building-a-tb-scale-m...

It kinda seems like a model trained on multiple languages would to some extent be better at English than a model trained only on English? I mean so much of English comes from other languages, and understanding language as a concept transcends any specific language. Of course there are limits and it needs good English vocabulary and understanding, but I feel the extra languages would help rather than hinder English performance.
Post reply on HN