Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OPT: Open Pre-trained Transformer Language Models

arxiv.org

Post reply on HN
Print view View original

181–190 of 242 posts

Re: OPT: Open Pre-trained Transformer Language Models

#181

Post by mdp2021 »

hoseja wrote: post #151
thorum wrote: post #31

A quick summary of the Limitations section: - "OPT-175B does not work well with declarative instructions or point-blank interrogatives." - "OPT-175B also tends to be repetitive and can easily get stuck in a loop. While sampling can reduce the incidence rate of repetitive behavior (Holtzman et al., 2020), we anecdotally found it did not eliminate it entirely when only one generation is sampled." - "We also find OPT-17…

At some point they have to face the reality these "stereotypical biases" are natural and hamstringing AIs to never consider them will twist them monstrously.

What about: at some point we would have to really catch that inspiration from the expression "Intelligence" and build a critical engine?

Edit: in fact, your latter statement seems to suggest finished products: no, they are toys. We are playing in order to build further, we are getting results, milestones in the construction abilities - but those "models" are little lab-byproducts monsters. What are you «twisting»?

Re: OPT: Open Pre-trained Transformer Language Models

#182

Post by hoseja »

wardedVibe wrote: post #173
Ar-Curunir wrote: post #160

Earlier quoted context omitted.

you're just saying "people are naturally racist" in more words.

They're saying that racist stereotypes are true, specifically.

No, I am saying that the cure is worse than the disease. The proper fix for the AI being racist is to make it able to not be racist on it's own (which would probably need much deeper understanding on the side of the AI), not forbid everything that passes some primitive heuristic of "being racist". One is painful and correct, the other is easy and feelgood and doomed.

Re: OPT: Open Pre-trained Transformer Language Models

#183

Post by spupe »

voz_ wrote: post #60
daenz wrote: post #57

Earlier quoted context omitted.

>You cannot fingerprint models like this A GAN can absolutely be trained to discriminate between text generated from this model or another model. >that's hilarious What's hilarious about it?

> A GAN can absolutely be trained to discriminate between text generated from this model or another model. Nope. I dare you to do it. Or at least intelligently articulate the model architectures for doing so. > What's hilarious about it? It's a bullshit term, firstoff, and calling yourself that is the height of ego. Might as well throw in rockstar, ninja, etc too.

> Nope. I dare you to do it. Or at least intelligently articulate the model architectures for doing so.

It is obvious that we can in principle try to detect this. People are already attempting to do so [1][2]. I would be very surprised if Facebook and other tech giants are not trying to do that, because they already have a huge problem in their hands from this type of technology.

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8049133/ [2] https://github.com/openai/gpt-2-output-dataset/tree/master/d...

Re: OPT: Open Pre-trained Transformer Language Models

#184

Post by andybak »

dhzhzjsbevs wrote: post #153
pyinstallwoes wrote: post #116

Earlier quoted context omitted.

Elon hasn’t been involved for 2+ years. Didn’t like the direction afaik.

Such a backfire on the narrative setup. Elons so evil amirite? He noped out of there when they started acting shady. Oof.

Gosh. You've seen right through us.

Re: OPT: Open Pre-trained Transformer Language Models

#185

Post by schleck8 »

MasterScrat wrote: post #2

"We are releasing all of our models between 125M and 30B parameters, and will provide full research access to OPT-175B upon request. Access will be granted to academic researchers; those affiliated with organizations in government, civil society, and academia; and those in industry research laboratories." GPT-3 Davinci ("the" GPT-3) is 175B. The repository will be open "First thing in AM" ( https://twitter.com/stephe…

Repo down?

Re: OPT: Open Pre-trained Transformer Language Models

#186

Post by __alexs »

coffeeblack wrote: post #134
Jensson wrote: post #107

Earlier quoted context omitted.

They gave a reason why they didn't release it to the public, they said it was too dangerous: https://www.theguardian.com/technology/2019/feb/14/elon-musk... But of course then they started selling it to the highest bidder, so I wouldn't really trust what they say. They aren't "OpenAI", at this point they are just regular "ProprietaryAI". I really wonder what goal Elon Musk have with it.

Didn’t Musk leave the organization because they started doing things he didn’t like?

His stated reason for leaving the board was potential future conflicts with things Tesla are working on.

Re: OPT: Open Pre-trained Transformer Language Models

#187

Post by wardedVibe »

hoseja wrote: post #182
wardedVibe wrote: post #173

Earlier quoted context omitted.

They're saying that racist stereotypes are true, specifically.

No, I am saying that the cure is worse than the disease. The proper fix for the AI being racist is to make it able to not be racist on it's own (which would probably need much deeper understanding on the side of the AI), not forbid everything that passes some primitive heuristic of "being racist". One is painful and correct, the other is easy and feelgood and doomed.

Fair enough, that's what I get for bringing reddit discussion norms with me.

Though because of how general purpose these models are, I have a hard time believing such a model couldn't be used to generate reams of racist screeds for propaganda/astroturfing purposes.

Re: OPT: Open Pre-trained Transformer Language Models

#188

Post by dhzhzjsbevs »

andybak wrote: post #184
dhzhzjsbevs wrote: post #153

Earlier quoted context omitted.

Such a backfire on the narrative setup. Elons so evil amirite? He noped out of there when they started acting shady. Oof.

Gosh. You've seen right through us.

Well, yea? You lot stopped caring about being seen long ago.

Re: OPT: Open Pre-trained Transformer Language Models

#189

Post by ChrisRR »

thorum wrote: post #31

A quick summary of the Limitations section: - "OPT-175B does not work well with declarative instructions or point-blank interrogatives." - "OPT-175B also tends to be repetitive and can easily get stuck in a loop. While sampling can reduce the incidence rate of repetitive behavior (Holtzman et al., 2020), we anecdotally found it did not eliminate it entirely when only one generation is sampled." - "We also find OPT-17…

Higher rate of toxicity and stereotypes?

So it was trained on facebook comments then

Post reply on HN