Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OPT: Open Pre-trained Transformer Language Models

arxiv.org

Post reply on HN
Print view View original

161–170 of 242 posts

Re: OPT: Open Pre-trained Transformer Language Models

#161

Post by mike_d »

ad_hominem wrote: post #155
mike_d wrote: post #146

Earlier quoted context omitted.

> Just look at the disastrous mess of his half-baked "opt-out" thing that flagrantly violates GDPR Pushshift collects data from Reddit using the same API as the mobile app and public site. It does not have any privileged access to the Reddit database, nor is it collecting any PII that would be subject to GDPR. You as a user grant a pretty broad license to Reddit when you post content. One of the things the license al…

> nor is it collecting any PII that would be subject to GDPR Yeah that's not how that works. Reddit is a free text input interface. I'm free to put PII in any post or comment I want to and you have to comply with data protection laws accordingly if I want my information redacted later on. The same way you wouldn't just "let it ride" if someone uploaded illegal content - the content itself is what's protected, doesn't…

That has already been hashed out in the European courts. The processor of the data needs to have a reasonable way of establishing that the data belongs to a identifiable natural person.

But by all means, if you disagree feel free to report Pushshift to the EU regulators. As far as I know Pushshift is based in the US and has no presence to establish a nexus to EU law.

Re: OPT: Open Pre-trained Transformer Language Models

#162

Post by IAmEveryone »

Gigachad wrote: post #145
tsol wrote: post #130

Earlier quoted context omitted.

Are there really no moderated forums that the data can be taken from? Even HN-based training data would be much more civil

A model trained on HN would spit out a 5 paragraph story about how minorities provide a negative ROI for cities. Or how the homeless need to removed from society.

Sure, but it would never do something actually bad, like raising the possibility that sexual harassment might, sometimes, be an issue, or questioning the value of phrenology.

Re: OPT: Open Pre-trained Transformer Language Models

#163

Post by IAmEveryone »

sanxiyn wrote: post #137
tsol wrote: post #130

Earlier quoted context omitted.

Are there really no moderated forums that the data can be taken from? Even HN-based training data would be much more civil

Note that HN is included in the training data, see page 20.

Go figure (8)!

Re: OPT: Open Pre-trained Transformer Language Models

#164

Post by can16358p »

Gigachad wrote: post #145
tsol wrote: post #130

Earlier quoted context omitted.

Are there really no moderated forums that the data can be taken from? Even HN-based training data would be much more civil

A model trained on HN would spit out a 5 paragraph story about how minorities provide a negative ROI for cities. Or how the homeless need to removed from society.

Don't forget that it must also generate, at some point regardless of the topic, a new terminal emulator, and an extremely positive or extremely negative opinion about how blockchain can solve a problem.

Re: OPT: Open Pre-trained Transformer Language Models

#165

Post by IAmEveryone »

robonerd wrote: post #9
javchz wrote: post #8

Earlier quoted context omitted.

My bet it's probably a filter, trying to prevent create a even more realistic farmbots in social media, as they are already bad as they are now.

But they'll consider requests from government and industry.. both greater threats in the information war than any private individual.

Since “everyone” would include governments and industry as well, their restriction is guaranteed to not contain more bad actors than no restriction.

Re: OPT: Open Pre-trained Transformer Language Models

#166

Post by hoseja »

IAmEveryone wrote: post #159
hoseja wrote: post #151

Earlier quoted context omitted.

At some point they have to face the reality these "stereotypical biases" are natural and hamstringing AIs to never consider them will twist them monstrously.

So if your plane model keeps blowing up, at some point people will just have to learn to live (/die) with it?

It's not blowing up though, it's experiencing natural turbulence and you're so afraid of getting jostled a bit you demand the plane be tethered to the ground and never exceed 10mph. How to fly under these conditions is left as an exercise for the reader.

Re: OPT: Open Pre-trained Transformer Language Models

#167

Post by fxtentacle »

ALittleLight wrote: post #5
MasterScrat wrote: post #2

"We are releasing all of our models between 125M and 30B parameters, and will provide full research access to OPT-175B upon request. Access will be granted to academic researchers; those affiliated with organizations in government, civil society, and academia; and those in industry research laboratories." GPT-3 Davinci ("the" GPT-3) is 175B. The repository will be open "First thing in AM" ( https://twitter.com/stephe…

I don't like "available on request". I just want to download it and see if I can get it to run and mess around with it a bit. Why do I have to request anything? And I'm not an academic or researcher, so will they accept my random request? I'm also curious to know what the minimum requirements are to get this to run in inference mode.

I'd guess they want to limit traffic. Once Huggingface links to you, your bandwidth bill 100x-es.

Re: OPT: Open Pre-trained Transformer Language Models

#168

Post by IAmEveryone »

dukeofdoom wrote: post #47
CrispinS wrote: post #44

Earlier quoted context omitted.

> The second link returned on him was from ADL. No way that's an organic result. It might be, actually. I understand why you'd think that, but look at the results for other search engines. Kagi: ADL in 2nd place Bing: ADL in 3rd place Yandex: ADL not on the first page, but SPLC[1] is the the 6th result [1]: https://www.splcenter.org/fighting-hate/extremist-files/indi...

This logic kind of fails quickly. I bet you wouldn't use it to show that Tiananmen Square did not happen, by showing all Chinese Search Engine are in apparent agreement on it not happening.

Does it often happen to you that you talk about Ai and, three minutes later, find yourself arguing with every search machine on the planet that it’s impossible that someone would say nasty things about your favorite fascist?

Re: OPT: Open Pre-trained Transformer Language Models

#169

Post by celdon25 »

throwawayohio wrote: post #123
ad_hominem wrote: post #96

Earlier quoted context omitted.

Not handy, and I'm not going to spend my evening digging. It may've also been one of the NGOs ideologically aligned with him that credited him for the data + assistance

If it's so egregious is it really that hard to find an example of the bias? Calling the integrity of a single person operation into question, but then backing out with no evidence and even saying it might not have even been them seems a bit irresponsible.

[flagged]

Re: OPT: Open Pre-trained Transformer Language Models

#170

Post by codebolt »

teaearlgraycold wrote: post #148
lumost wrote: post #37

I often wonder if OpenAIs decision not to open gpt-3 was because it was to expensive to train relative to its real value. They’ve hidden the model behind an api where they can filter out most of the dumb behaviors, while everyone believes they are working on something entirely different.

> They’ve hidden the model behind an api where they can filter out most of the dumb behaviors What do you mean by this?

Things like cobbling on a bunch of heuristic rule-based behaviours that wouldn't look good in the public repo of a supposed quasi-AGI system?
Post reply on HN