Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

471–480 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#471

Post by xcambar »

bgroat wrote: post #458
AlecSchueler wrote: post #416

Earlier quoted context omitted.

On average, it doesn't. This is why advertising and magic work.

I'm a magician and a developer by training. Now primarily employed in a marketing capacity. Over my career I've worked with: - Doctors - Lawyers - Engineers - Fund managers - Academics (hard and soft sciences) - Mentalists/Hypnotists All of them believed that they're specific training and temperament made them immune from simple persuasion techniques and that they were purely rational actors. None of them struck me a…

It is typical to rate yourself above your actual self.

Even when someone rates oneself down like when saying of themself that they're dumb, ugly or whatever, they generally mean it in a lesser fashion than for any other peer they'd attribute as such.

Re: YaLM-100B: Pretrained language model with 100B parameters

#472

Post by f311a »

mumblemumble wrote: post #460
lumost wrote: post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

From what I've seen, using these huge models for inference at any kind of scale is expensive enough that it's difficult to find a business case that justifies the compute cost.

Yandex uses it for search and voice assistant

Re: YaLM-100B: Pretrained language model with 100B parameters

#473

Post by melenaboija »

thejosh wrote: post #65
ketzu wrote: post #21

Seeing those gigantic models it makes me sad that even the 4090 is supposed to stay at 24GB of RAM max. I really would like to be able to run/experiment on larger models at home.

It's also a power issue. The 4090 sounds like you're going to need a much, MUCH higher PSU than you currently use.. or it'll suddenly turn off as it uses 2-3x the power. You'll need your own wiring to run your PC soon :-)

I think it is a stupid question, but does the power consumption needed by processors to infer compared to human brains demonstrate that there is something fundamentally wrong for the AI approach or is it more physics related?

I am not a physicist or biologist or anything like that so my intuition is probably completely wrong but it seems to me that for more basic inference operations (lets say add two numbers) power consumption from a processor and a brain is not that different. It’s like seeing how expensive it is for computers to infer for any NLP model, humans should be continuously eating carbs just to talk.

Re: YaLM-100B: Pretrained language model with 100B parameters

#474

Post by gambler »

lostmsu wrote: post #470
gambler wrote: post #419

Earlier quoted context omitted.

Blatantly incorrect. Google engages in egregious political censorship all the time. Including censorship for Russian government and censorship of US anti-war voices. https://reclaimthenet.org/youtube-responds-to-cpac-censorshi... https://reclaimthenet.org/google-expanded-its-censorship-of-... https://reclaimthenet.org/russia-continues-to-order-google-t... In US they pretend to "decide" to censor things "on their own"…

None of your links show Google censoring anti-war propaganda.

https://medium.com/dan-sanchez/don-t-see-evil-148ae18bc9fe

https://citizenactionmonitor.wordpress.com/2017/08/02/google...

https://www.protocol.com/bulletins/google-censors-war

Re: YaLM-100B: Pretrained language model with 100B parameters

#475

Post by lostmsu »

dang wrote: post #431
gambler wrote: post #419

Earlier quoted context omitted.

Blatantly incorrect. Google engages in egregious political censorship all the time. Including censorship for Russian government and censorship of US anti-war voices. https://reclaimthenet.org/youtube-responds-to-cpac-censorshi... https://reclaimthenet.org/google-expanded-its-censorship-of-... https://reclaimthenet.org/russia-continues-to-order-google-t... In US they pretend to "decide" to censor things "on their own"…

> This is either astounding ignorance or blatant gaslighting. Can you please edit name-calling / swipes like that out of your HN comments? It breaks the site guidelines and weakens your point. https://news.ycombinator.com/newsguidelines.html

Considering the importance of the topic, and provided the linked articles actually contained examples of Google censoring anti-war propaganda, I believe the swipe would have been fully justified.

Highly emotional tone changes how the data affects the reader. If he is right, I would surely better remember next time that Google is in the same ballpark due to the insult hitting hard. If he is wrong, I will know better to ignore such claims in the future without a direct quote or something else that consumes less time than reading an entire linked article.

Re: YaLM-100B: Pretrained language model with 100B parameters

#476

Post by mardifoufs »

varispeed wrote: post #435
aaaaaaaaata wrote: post #177

Earlier quoted context omitted.

Wait until you hear about the folks your country is killing!

https://en.wikipedia.org/wiki/And_you_are_lynching_Negroes

Yes posting that Wikipedia link isn't a magic way to deflect from the fact that the iraq war led to a million people dead. And that people are still dying from the war on terror. It's amazing that you just said that people in ukraine are still dying, and that just saying that you don't support your government from the comfort of your couch isn't enough... and then you proceeded to link an article specifically so that you can ignore/deflect the deaths that are also happening now and that should be (according to your own argument) much more important than any of your own comfort or even liberty?

"Yes hundreds of thousands of Muslims died and are still dying, but bringing it up or asking me to do anything about is fallacious! Checkmate"

As you said, who cares about debate tricks when people in the middle east are still dying from the war on terror as we spead? Why are you holding other people to standards that you don't even pretend to hold yourself to? You are expecting people to get arrested to prevent deaths and talk about the situation in ukraine, but I guess making you uncomfortable with "whataboutism" is the limit?

Re: YaLM-100B: Pretrained language model with 100B parameters

#477

Post by codeviking »

wskinner wrote: post #466
codeviking wrote: post #463

Earlier quoted context omitted.

Which is why it's important for folks to start applying AI to more interesting (but harder, more nuanced) problems. Instead of making it easier for people to write emails, or targeting ads, it should be used to help doctors, surgeons and scientists. The problem is that these problems are less profitable. And that the companies with enough compute to train these types of models are concerned about getting more eyeball…

The problem is not that those problems are less profitable. The problem is a combination of 1. Those problems are much harder 2. The potential harm from getting them wrong is much larger

Yup, I definitely agree that they're harder (and noted this). But I'm not sure I agree with your second point. Or rather, I think there's some nuance to it.

Sure, using AI to treat people without a human in the loop would clearly do harm. But using AI as an assistant, to help a doctor make the right diagnosis, seems like it'd do the opposite. It'd help doctors serve a larger patient population, make less mistakes, and probably equate to less harm in the long run.

Anyway, I think we can all agree that using AI for anything other than ad targeting is a net win.

Re: YaLM-100B: Pretrained language model with 100B parameters

#478

Post by gfodor »

lumost wrote: post #456

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

An equally plausible frame is that once a technology becomes replicated across several companies, it makes sense to open source it since the marginal competitive advantage are the possible resultant external network effects.

I don't know if that's the right way to think about the open sourcing of large language models. I just think we really can't read too much into such releases regarding their motivation.

Re: YaLM-100B: Pretrained language model with 100B parameters

#479

Post by lostmsu »

gambler wrote: post #474
lostmsu wrote: post #470

Earlier quoted context omitted.

None of your links show Google censoring anti-war propaganda.

https://medium.com/dan-sanchez/don-t-see-evil-148ae18bc9fe https://citizenactionmonitor.wordpress.com/2017/08/02/google... https://www.protocol.com/bulletins/google-censors-war

Still no. First case I would not even consider censorship. The third one was temporary until Google stopped operating in Russia altogether.

A quote from the second one: "cumulative 45 percent decrease in traffic from Google searches"

Re: YaLM-100B: Pretrained language model with 100B parameters

#480

Post by mardifoufs »

lotusmars wrote: post #170
dmpk2k wrote: post #166

Earlier quoted context omitted.

And yet it is still true.

Americans just love to talk about themselves. Who cares about Russians under Putin's oppression or Ukrainians being exterminated. Let's talk about your government, Bush, Trump and Google.

This is not true. I can assure you that tons of Muslims and people from the middle east also care about the fact that the same actors who gleefully engineered wars on terror that led to a million people dying and entire countries getting devastated, with absolutely 0 consequences for them, are now so very keen to hold other people accountable for illegitimate invasions.

No one likes hypocrisy, especially when it is coming from the same westerners that at most protested for a few weeks back in 2003 when their own countries bombed us for 2 decades, that are now calling for other people to get arrested and possibly tortured/executed by putin's regime because that's just the right thing(tm) to do to stop the war. It would be laughable if it wasn't despicable.

Post reply on HN