Live data from Hacker News

OPT: Open Pre-trained Transformer Language Models

arxiv.org

191–200 of 242 posts

Re: OPT: Open Pre-trained Transformer Language Models

#191
post #180

Just curious, will I be able to use it using my Nvidia card with 10GB of memory? Does it require multiple graphic cards?

As the model weights (even quantized) would be several hundred GBs, it’s unlikely, unless special inference code is written that loads and processes only a small subset of weights and calculations at a time. But running it that way would be painfully slow.

Re: OPT: Open Pre-trained Transformer Language Models

#192
post #57
post #55

Earlier quoted context omitted.

> I will never discuss FB technical details, internals, or anything else on this site, so please do not ask. My claim of nonsense has nothing to do with FB. You cannot fingerprint models like this, that's just not how it works. Also, if we are reading profiles, you call yourself a 10x engineer on your blog, that's hilarious. Maybe 10x the nonsense?

>You cannot fingerprint models like this A GAN can absolutely be trained to discriminate between text generated from this model or another model. >that's hilarious What's hilarious about it?

A well trained GAN has 50% chance of finding if the generate image is fake or not. But you can't do imperceptible changes on text like you for images.

Re: OPT: Open Pre-trained Transformer Language Models

#193
post #65
post #60

Earlier quoted context omitted.

> A GAN can absolutely be trained to discriminate between text generated from this model or another model. Nope. I dare you to do it. Or at least intelligently articulate the model architectures for doing so. > What's hilarious about it? It's a bullshit term, firstoff, and calling yourself that is the height of ego. Might as well throw in rockstar, ninja, etc too.

So in the entire field of machine learning, we can't train a model that can identify another model from its output? Just can't be done? And there's absolutely no value in having tools that can identify deep fakes, or content produced by specific open models? >It's a bullshit term, firstoff, and calling yourself that is the height of ego I am a 10x engineer though, so I'm sorry if that rubs you the wrong way. Also, yo…

> in the entire field of machine learning

... we can't train a model to be 100% correct. There will always be false matches. Another super hard task is confidence estimation - models tend to be super sure of many bad predictions.

In this particular case you're talking about detecting human written texts against stochastic text generation. If you wanted to test if the model regurgitates training data, that would have been easy. But the other way around, to check if it outputs something different from future text, it's a hard, open-ended problem. Especially if you take into consideration the prompts and the additional information they could contain.

It's like testing if I have my keys in the house vs testing if my keys are not outside the house (can't prove an open ended negative). On top of this, the prompts would be like allowing unsupervised random strangers into the house.

Re: OPT: Open Pre-trained Transformer Language Models

#196
post #114

Remember when OpenAi wrote this? > Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT-2 along with sampling code. We are not releasing the dataset, training code, or GPT-2 model weights Well I guess Meta doesn’t care. https://openai.com/blog/better-language-models/

OpenAI is only concerned with making money. What you quote is the PR reason, so they don't sound like the empty corporate money-grubbers they actually are.

Re: OPT: Open Pre-trained Transformer Language Models

#198
post #132

Earlier quoted context omitted.

Sure, but I'm an individual and free to say what I do and don't like. Why is that hard to understand?

Because it's a dumb thing to say. "Not really a fan of having to pay for my dinner!" It's just silly.

What's wrong with thinking that a society should provide for the basic needs of its members?

Re: OPT: Open Pre-trained Transformer Language Models

#200

Earlier quoted context omitted.

Gosh. You've seen right through us.

Well, yea? You lot stopped caring about being seen long ago.

Both your comments indicate that you regard everyone here as some kind of homogonous group who share the same views - whilst you are somehow outside or different.

That's a bit like sitting in a traffic jam complaining about the other cars. You are one of us and probably not a huge outlier either in most regard.

I don't know why you have ended up with a me vs them perception but it's probably fairly unhealthy and I hope it's something you carry around in real life as well.

Post reply on HN