Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

YaLM-100B: Pretrained language model with 100B parameters

github.com

Post reply on HN
Print view View original

321–330 of 666 posts

Re: YaLM-100B: Pretrained language model with 100B parameters

#321

Post by waffleiron »

londons_explore wrote: post #214
ptnxlo wrote: post #98

Earlier quoted context omitted.

Care to elaborate?

There is lots of content Google bans/hides. Copyrighted content, Adult content, child pornography, official secrets, etc. I don't think thats so different from other countries which also have a (partially overlapping) list of whats not allowed. Normally, when people think about that they say "well pictures of naked children are morally wrong, whereas talking about LGBTQ stuff is fine". But people in other parts of th…

Also google specifically bans content that:

- Disparage or belittle victims of violence or tragedy.

- Deny an atrocity.

- We don’t allow content that promotes terrorist or extremist acts, which includes recruitment, inciting violence, or the celebration of terrorist attacks.

Now I don't think these are bad rules, but they are rules that very much depend on the official narrative. A terrorist to one is a freedom fighter to another. These are rules that can be applied as wanted.

https://support.google.com/websearch/answer/10622781

Re: YaLM-100B: Pretrained language model with 100B parameters

#322

Post by nosianu »

afroboy wrote: post #265
bgandrew wrote: post #155

Earlier quoted context omitted.

no it's not. they straight up serve kremlin, promoting kremlin fake news and silencing russian opposition (not much to silence but still). they can have whatever functionality they like, I still won't use it in billion years.

Literally what google doing in favor of USA.

I doubt that anything like this happend to Google execs in the US:

"Putin's agents reportedly threatened a top Google executive in Moscow with a 24-hour ultimatum – Take down Russia protest vote app or go to prison" -- https://www.businessinsider.com/russia-agents-threatened-goo...

Not yet at least, the political climate may deteriorate to that point, especially when it's about elections, given recent revelations.

Still, at least right now it looks to me - and I have visited Russia and Ukraine several times in the past and still have indirect connections (to people heavily involved in business there) - that there still is considerable more freedom from the government and its wishes for people and companies in the West.

If you publicly criticize a US politician you may get some hate messages, but at least they are from private citizens and you don't have FBI agents knocking on your door threatening you with prison. In Germany some rogue police were found to send threatening messages, but as soon as it was discovered the government acted against it. Also in Germany there even were public rallies from pro-Russian folks, now try that in Moscow with pro-Ukraine banners... Russia even bans the colors yellow and blue, even when they have nothing whatsoever to do with Ukraine and are just decorative: "Russians Strip Yellow and Blue From the Nation’s Streets Over Ukraine War" -- https://www.themoscowtimes.com/2022/04/27/in-photos-russians...

Re: YaLM-100B: Pretrained language model with 100B parameters

#323

Post by lostmsu »

toxik wrote: post #254
amelius wrote: post #52

Earlier quoted context omitted.

It's not really random access. I bet the graph can be pipelined such that you can keep a "horizontal cross-section" of the graph in memory all the time, and you scan through the parameters from top to bottom in the graph.

Fair point, but you’ll still be bounded by disk read speed on an SSD. The access pattern itself matters less than the read cache being << the parameter set size.

Top SSDs do over 4GB/s so you can infer in 50 seconds if disk bound.

You can also infer a few tokens at once, so it will be more than 1 char a minute. Probably more like sentence a minute.

Re: YaLM-100B: Pretrained language model with 100B parameters

#324

Post by lostmsu »

yarandex wrote: post #95
Aardwolf wrote: post #20

Earlier quoted context omitted.

Way too slow on CPU unfortunately But this does make me wonder if there's any way to allow a graphics card to use regular RAM in a fast way? AFAIK built-in GPU's inside CPU's can but those GPU's are not powerful enough

Assuming running on CPU is memory-bandwidth limited, not CPU-limited, it should take about 200GB / (50GB/sec) = 4 seconds per character. Not too bad.

That's per token. And you can generate quite a few per pass.

Re: YaLM-100B: Pretrained language model with 100B parameters

#325

Post by adamc »

ggktk wrote: post #272
sanxiyn wrote: post #261

Earlier quoted context omitted.

They can (and do) revoke API access from bad guys. They can't do that to downloaded models. Look, I don't like what OpenAI does, but "API access, but no model download" makes sense if you are worried about misuses.

Bad actors still can get access to such models. It even makes them more dangerous than it would if everyone had access to them. Here's an alternative: progressively release better and better models (like 3B params, 10B, 50B, 100B) and let people figure out the best way to fight against bad actors using them.

"It even makes them more dangerous..." needs to be demonstrated, not asserted.

Re: YaLM-100B: Pretrained language model with 100B parameters

#326

Post by f6v »

pastacacioepepe wrote: post #165
toyg wrote: post #45

Earlier quoted context omitted.

Well... I'm sorry if I reach for the reductio at Hitlerum, but any achievements Nazi scientists might have reached in concentration camps are definitely tainted. Similarly, achievements in the field of online consumer analysis in a country where consumer-privacy protections are nonexistent, surely should be considered tainted...?

Wow. Yes you should have refrained from this. You are comparing Nazi scientists who killed many innocents to some software engineers working on a cool project and releasing it for free to the world. What is your problem?

Just FYI regardless of your stance on any of the recent conflicts. De-humanization is a primary tool in information warfare these days.

Re: YaLM-100B: Pretrained language model with 100B parameters

#327

Post by FpUser »

lotusmars wrote: post #145
FpUser wrote: post #141

Earlier quoted context omitted.

>""Oh yes I don't support my government, but you know these arrests, I'd rather stay in my cosy home and enjoy my tea." Why are you so surprised? This is exactly how most of the population behaves everywhere. People go about their business and "support" criminal actions of their governments all the time. This includes the West. Our governments have no problems exterminating, starving and displacing people (as long as…

> Our governments have no problems exterminating, starving and displacing people It's high time to bring "but US bombed Iraq". Classic playbook.

It is as classic as your own standard script. I've just explained what is going on. I did not want to single out the US as it happens everywhere. But if you are so touchy maybe you should not have "supported" that particular subject. Remind me what was your punishment?

Re: YaLM-100B: Pretrained language model with 100B parameters

#328

Post by waffleiron »

riku_iki wrote: post #313
waffleiron wrote: post #306

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/PRISM#The_slides

There is no clarity on these slides if collection happened proactively or it was a way to transfer information for FISA warrants.

You asked for proof of the following:

> Google and Facebook feed their data to NSA.

We know that at least some companies were ordered to handover all data, continuously [1].

edit: I think we have enough evidence that I would assume that it's valid for the other companies on the slides, and if it's not true you'll have to provide some proof of that.

edit 2: [2]

> It searches that database and lets them listen to the calls or read the emails of everything that the NSA has stored, or look at the browsing histories or Google search terms that you've entered, and it also alerts them to any further activity that people connected to that email address or that IP address do in the future."

> Greenwald explained that while there are "legal constraints" on surveillance that require approval by the FISA court, these programs still allow analysts to search through data with little court approval or supervision.

> "There are legal constraints for how you can spy on Americans," Greenwald said. "You can't target them without going to the FISA court. But these systems allow analysts to listen to whatever emails they want, whatever telephone calls, browsing histories, Microsoft Word documents."

> "And it's all done with no need to go to a court, with no need to even get supervisor approval on the part of the analyst," he added.

edit 3:

> Equally unusual is the way the NSA extracts what it wants, according to the document: “Collection directly from the servers of these U.S. Service Providers: Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.” [3]

[1] https://www.theguardian.com/world/2013/jun/06/nsa-phone-reco...

[2] https://abcnews.go.com/blogs/politics/2013/07/glenn-greenwal...

[3] https://www.washingtonpost.com/investigations/us-intelligenc...

Re: YaLM-100B: Pretrained language model with 100B parameters

#329

Post by nashashmi »

yonixw wrote: post #287
VictorPath wrote: post #274

Earlier quoted context omitted.

The relevant Wikipedia page is https://en.wikipedia.org/wiki/Khazars

Did you read it? It literally has a "Use in antisemitic" section. Can you have any bigger red flag? > Use in antisemitic polemic > conspiracy theorist, David Icke, who states that the Israelians falsely claim to be descendants of the Biblical Jews I don't really care about conspiracy theorists. Mainly because they ignore 2000 years of accepted archeology.

https://en.wikipedia.org/wiki/The_Thirteenth_Tribe#Genetic_r...

This led me to look up similar information. Another article [1] looks into this a little more deeply.

I feel there is a resurgence of despising European dominance over the last 200 years and Israel is just another point here. Thus, we have material hypothesizing the illegitimacy of European Jews when the Jews of other ethnicities may have better acceptance in the region. (But all of this is just a vague hypothesis.)

[1] https://www.science.org/content/article/tracing-roots-jewish...

Re: YaLM-100B: Pretrained language model with 100B parameters

#330

Post by tablespoon »

323 wrote: post #235
hdjjhhvvhga wrote: post #175

Earlier quoted context omitted.

> Google overlords neutered their own product out of fear over lawyers/regulation What kind of lawyers/regulation do you have in mind? If anything, I'd find the opposite: lawyers and copyright holders should be grateful for such a tool that - when it was still working - allowed you to trace websites using your images illegally. Now they all use Yandex for this purpose, with relatively good results.

You misunderstood parent post. It's about Google not being sued for discrimination. https://www.washingtonpost.com/news/the-intersect/wp/2016/08... https://www.theguardian.com/technology/2016/apr/08/does-goog... https://www.bloomberg.com/news/articles/2021-10-19/google-qu... https://theconversation.com/googles-algorithms-discriminate-...

> You misunderstood parent post. It's about Google not being sued for discrimination.

Who's suing them and on what grounds? If they made changes, it's probably for PR reasons, not legal ones.

Also not all of these seem "fixed" e.g.:

> https://www.theguardian.com/technology/2016/apr/08/does-goog...

Article from 2016, but results look very similar today: https://www.google.com/search?q=unprofessional+hair&source=l...

Post reply on HN