Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

621–630 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#621

Post by raducu »

HeavyStorm wrote: post #492
lumost wrote: post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

Maybe training it is not that expensive? I know from practice that it takes a really really long time to train even a small nn (thousands of params) , so you'll need a lot more hardware to train one with billions... But, it's expensive to buy the hardware, not necessarily to use it. If you, for some reason, have a few hundred GPU lying around, it might be "cheap" to do the necessary training. Now, that's not your poi…

> If you, for some reason, have a few hundred GPU lying around

Not to nitpick, but that is like saying that if you have a Lamborghini lying around, a Sunday trip in one is not so expensive.

Re: YaLM-100B: Pretrained language model with 100B parameters

#622

Post by nosianu »

waffleiron wrote: post #337
nosianu wrote: post #322

Earlier quoted context omitted.

I doubt that anything like this happend to Google execs in the US: "Putin's agents reportedly threatened a top Google executive in Moscow with a 24-hour ultimatum – Take down Russia protest vote app or go to prison" -- https://www.businessinsider.com/russia-agents-threatened-goo... Not yet at least, the political climate may deteriorate to that point, especially when it's about elections, given recent revelations. St…

> you don't have FBI agents knocking on your door threatening you with prison. Correct, it's DHS. https://twitter.com/_secondthought/status/133274617257067725...

You can twist it however you like, and select individual sentences and ignore the context and everything else I wrote. At this point (again, may get worse, it does not look good IMO) Russia is at least an order of magnitude worse. You have individual cases - maybe - in the US, but in Russia it's systemic and systematic and goes all the way to murder not just of the person but of the entire family - as we saw in March, when two oligarchs and their entire family were murdered, one in Russia, one in Spain (summary of oligarch deaths: https://www.businessinsider.com/these-are-all-the-russian-ol...). On top of that various murders of "traitors" e.g. in UK and in Germany. Trump got close in rhetoric to what Putin said on Russian TV in March but did not have a chance to implement it, and the US institutions still resist and don't (yet?) follow a dictator blindly, as they do in Russia.

Re: YaLM-100B: Pretrained language model with 100B parameters

#623

Post by [deleted] »

codeviking wrote: post #477
wskinner wrote: post #466

Earlier quoted context omitted.

The problem is not that those problems are less profitable. The problem is a combination of 1. Those problems are much harder 2. The potential harm from getting them wrong is much larger

Yup, I definitely agree that they're harder (and noted this). But I'm not sure I agree with your second point. Or rather, I think there's some nuance to it. Sure, using AI to treat people without a human in the loop would clearly do harm. But using AI as an assistant, to help a doctor make the right diagnosis, seems like it'd do the opposite. It'd help doctors serve a larger patient population, make less mistakes, an…

[deleted]

Re: YaLM-100B: Pretrained language model with 100B parameters

#624

Post by trhway »

pastacacioepepe wrote: post #620
lotusmars wrote: post #222

Earlier quoted context omitted.

Say what you want, most people in Eastern Europe, Ukraine and even lots of people in Russia would prefer to be under protection of NATO. If it were not for NATO, Russian rapists would already be in Tallinn, Vilnius, Helsinki. Claiming they were offended by historical injustices therefore women should be raped and men shot dead ("denazified").

> Say what you want, most people in Eastern Europe, Ukraine and even lots of people in Russia would prefer to be under protection of NATO. Not the people in Donbass and Crimea apparently. > Russian rapists Oh please, quit the blatant Lyudmila Denisova propaganda. She was fired by your own government for all the falsehood she spread.

Unfortunately firing Lyudmila Denisova doesn't unrape all the victims of all those mass rapes by Russian soldiers in Ukraine. She was doing propaganda and Ukraine rightly fired her for that.

Russian soldiers have been shooting civilians at will (it is called "safari" in Russian army, and those Bucha streets - Yablunska and Vokzalnaya - with the bodies lying along the street is an example of such a "safari"). That would be an unique miracle in history if the soldiers wouldn't rape given such power in such conditions (for example those mass rapes by Soviet soldiers - 2 million of women were raped - back then in Germany in 1945 illustrate a result of such a power by the occupying force). And there is no miracle - just google Ukrainian rapes.

Re: YaLM-100B: Pretrained language model with 100B parameters

#625

Post by danuker »

5e92cb50239222b wrote: post #176
FpUser wrote: post #125

Earlier quoted context omitted.

>"It just strikes me how everyone ..." Because most of those "everyone's" are not facing the choices themselves and are basically keyboard warriors. Let's see what they say when they'll be asked to sacrifice their own well being to be on a "high moral ground".

I think we're already seeing that in recent French elections. I have a feeling this is only the beginning. As a relatively neutral party in all of this whose country hasn't tainted itself (but who nevertheless spent all my live under a similar autocracy), I can't help but shake my head at keyboard revolutionaries who definitely would have overthrown the regime, if only they lived in Russia. You just have no freaking…

> Guard your democracy as best you can

If you look at Snowden and Assange, that ship has sailed.

Re: YaLM-100B: Pretrained language model with 100B parameters

#626

Post by xpl »

The_Colonel wrote: post #616
xpl wrote: post #581

Earlier quoted context omitted.

> Russia can't sustain waging wars if they don't have an economy and disgruntled population. Russia can wage wars on natural resource selling alone, it only needs to keep the gas and oil flowing through the infrastructure. All those private companies' activities the government sees mostly as a distraction, it doesn't give a damn about them (until they get in the way). They don't matter much. It's very much unlike the…

> Russia can wage wars on natural resource selling alone, it only needs to keep the gas and oil flowing through the infrastructure. With sanctions it can't. > So if Yandex leaves the scene, the only game in town would be state-owned. This only reinforces the evil regime. It doesn't. Nationalization of companies rarely works well. Even then they can be hurt, made unprofitable, forcing the evil regime to divert resourc…

> Nationalization of companies rarely works well

It does not need to "go well". All the government needs is control. They have VK (controlled by Putin's people). They will be happy with either outcome with Yandex — kill Yandex entirely, and people will just switch to using VK services. Make Yandex controlled by the government, and then its resources will be used for evil purposes. So either way it favors the government, reducing overall freedom.

> It has no value

It has the value. I insist that the net profit from "killing Yandex" is strictly negative, as it has not even zero effect on preventing actual people from getting killed — the actual effect would be the other way around.

> With sanctions it can't.

Unfortunately, not true. Even if Europe stops buying Russian resources today, the remaining profits from selling to China, India and etc. would cover the expenses. The oil and gas prices will greatly rise (it already happens), so that would compensate the losing of the European markets.

Re: YaLM-100B: Pretrained language model with 100B parameters

#627

Post by guerrilla »

chupasaurus wrote: post #511
guerrilla wrote: post #181

Earlier quoted context omitted.

And what if they hadn't?

"Couldn't be penalized whatsoever" wasn't enough? In broader view, government could start taking hostages like they did with Google last December, but that wouldn't help reaching the goal even a small bit back then since there was no leverage on technical side of things.

> "Couldn't be penalized whatsoever" wasn't enough?

No, I think that's very naive.

Re: YaLM-100B: Pretrained language model with 100B parameters

#628

Post by orbital-decay »

shakow wrote: post #617
orbital-decay wrote: post #605

Earlier quoted context omitted.

>some of the Yandex products are actually better and pretty innovative (ignoring the political stuff). Maps and Go are especially good. Yeah, the same Yandex Maps that stopped showing state borders recently, as they are now "more focused on natural objects", in their words.

At the same time, from a consumer perspective, this sacrifice to internal political pressures won't hamper your usage of the product. It's not like if international borders were the most important annotation on a map for everyday use. Edit: in fact, I just checked, yandex maps still shows state borders.

It doesn't distinguish state and region borders anymore, like it did before. No borders on the overview map either. Just zoomed into a random place on the map, and I can tell where Turkey ends and Bulgaria begins only because the city names are different.

Re: YaLM-100B: Pretrained language model with 100B parameters

#629

Post by lairv »

MasterScrat wrote: post #570
lumost wrote: post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

HuggingFace will soon release their BigScience model: https://twitter.com/BigScienceLLM/status/1539941348656168961 "a 176 billion parameter transformer model that will be trained on roughly 300 billion words in 46 languages" So anything smaller than that will become worthless. May be a factor, companies have a last chance to make a PR splash before it happens. Read more about it: https://bigscience.huggingface.co/blo…

"worthless" huh, not everyone can afford inference of a ~500gb models, depending on the the speed/rate you need you might definitely go for smaller model

But maybe your sentence was more about "after BigScience model, open-sourcing anything smaller than that will be useless" which isn't necessarily true either, because there is still room to improve parameter efficiency, i.e. smaller models with comparabale performances

Re: YaLM-100B: Pretrained language model with 100B parameters

#630

Post by jokethrowaway »

1024core wrote: post #598
stainablesteel wrote: post #590

Earlier quoted context omitted.

This is a weird comment because yes, that's exactly what the above person was saying. It shows results that google won't give you. Secondly, I've yet to see any criticisms of 2000 mules data that aren't addressed by the stringency in the analysis they claim to have done. I thought the information they presented was extremely valuable. Are we going to overturn an election at this point? No. But the vulnerabilities to…

> then clearly taken advantage of. If you have any evidence, any evidence at all, of significant mail-in ballots fraud, then you should write it up and publish it; and even present it to the USDOJ, because you would have succeeded where Trump's highly-paid teams of lawyers failed. If you don't have proof, then please STFU.

It's not about whether it happened or not, it's whether it's reported or not.

I personally believe (with obviously no proof) there was definitely fraud going on, on both sides. With such an archaic system and such a great economic and power incentive, you would be stupid not to do it. For sure mail in ballots made it even easier than in the past.

I've heard about Russian hacking the elections after Trump won for a good 2 years.

Post reply on HN