Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OPT: Open Pre-trained Transformer Language Models

arxiv.org

Post reply on HN
Print view View original

211–220 of 242 posts

Re: OPT: Open Pre-trained Transformer Language Models

#211

Post by levesque »

saynay wrote: post #205
ALittleLight wrote: post #5

Earlier quoted context omitted.

I don't like "available on request". I just want to download it and see if I can get it to run and mess around with it a bit. Why do I have to request anything? And I'm not an academic or researcher, so will they accept my random request? I'm also curious to know what the minimum requirements are to get this to run in inference mode.

If it is like many other models, part of the reason would just be to reduce their bandwidth costs. The models can be huge, and they want to limit those who just want to download it on a whim so they don't rack up $10k+ is bandwidth charges, as has happened to many others who hosted big models out on S3 or something.

If only there was a way to distribute large files in a peer-to-peer manner, thus reducing the load on facebook's servers to effectively nothing. That would likely result in a torrent of bits being shared without any issues!

Re: OPT: Open Pre-trained Transformer Language Models

#212

Post by robbedpeter »

ad_hominem wrote: post #56
thorum wrote: post #31

A quick summary of the Limitations section: - "OPT-175B does not work well with declarative instructions or point-blank interrogatives." - "OPT-175B also tends to be repetitive and can easily get stuck in a loop. While sampling can reduce the incidence rate of repetitive behavior (Holtzman et al., 2020), we anecdotally found it did not eliminate it entirely when only one generation is sampled." - "We also find OPT-17…

> Pushshift.io Reddit corpus Pushshift is a single person with some very strong political opinions who has specifically used his datasets to attack political opponents. Frankly I wouldn't trust his data to be untainted. These models really need to be trained on more official data sources, or at least something with some type of multi-party oversight rather than data that effectively fell off the back of a truck. edit…

Web scraping is legal. Reddit users, like all other members of public forums, put their comments on the internet for the whole world to see. And collect, parse, process and manipulate. If you don't want the whole world to have access to your writing, you'd have to join a private forum.

Trying to shoehorn social media posts into some contorted post-hoc bastardization of the concept of privacy is ridiculous.

Shockingly, things that people post to publicly accessible websites are accessible by the public. We're starting to see social damage from this, with facial recognition and authoritarian governments using people's posts for tracking and oppression.

Decentralized services with strong legislation protecting personal data, and globally recognized content licensing will all be needed to prevent future abuse, but everyone currently in the planet over the age of 20 is more or less personally responsible for the massive and naive oversharing. We know better now, but 15+ years ago nobody except Sci-fi authors and fringe activists had a grasp of how badly unprotected globally shared streams of consciousness could go wrong.

Re: OPT: Open Pre-trained Transformer Language Models

#213

Post by robbedpeter »

f311a wrote: post #180

Just curious, will I be able to use it using my Nvidia card with 10GB of memory? Does it require multiple graphic cards?

The smaller models, yes. I'd bet dollars to donuts that gpt-neo and EleutherAI models outperform most, if not all, of Facebook's.

Check out huggingface, you'll be able to run a 2.7b model or smaller.

https://huggingface.co/EleutherAI/gpt-neo-2.7B/tree/main

Re: OPT: Open Pre-trained Transformer Language Models

#214

Post by orbital-decay »

PeterisP wrote: post #87
p1esk wrote: post #43

Earlier quoted context omitted.

How can you identify content generated with them?

I'm not saying that Meta did it, but recent research shows that it is possible and hard to detect - https://arxiv.org/abs/2204.06974 - so if they really wanted to, they could.

That paper is not about fingerprinting the arbitrary output of a specific model, which would allow Meta to track its usage in the results, e.g. tell a genuine text from a fake generated by their model. The paper implies giving the model some specific secret input only known to you.

I think the thread we're in is also based on the similar misunderstanding.

Re: OPT: Open Pre-trained Transformer Language Models

#215

Post by robbedpeter »

etaioinshrdlu wrote: post #35

What type of hardware would you need to run it?

A cluster of many $8000+ gpus. You're looking at around 350GB of vram, so 30 12gb gpus - a 3090 will cost around $1800, so $54k on the gpus, probably another $15k in power, cooling, and infrastructure, $5k in network, and probably another $20k in other costs to bootstrap it.

Or wait 10 years, if gpu capacity scales with Moore's law, consumer hardware should be able to run a ~400GB model locally.

Re: OPT: Open Pre-trained Transformer Language Models

#216

Post by JoeyBananas »

ALittleLight wrote: post #5
MasterScrat wrote: post #2

"We are releasing all of our models between 125M and 30B parameters, and will provide full research access to OPT-175B upon request. Access will be granted to academic researchers; those affiliated with organizations in government, civil society, and academia; and those in industry research laboratories." GPT-3 Davinci ("the" GPT-3) is 175B. The repository will be open "First thing in AM" ( https://twitter.com/stephe…

I don't like "available on request". I just want to download it and see if I can get it to run and mess around with it a bit. Why do I have to request anything? And I'm not an academic or researcher, so will they accept my random request? I'm also curious to know what the minimum requirements are to get this to run in inference mode.

A 175B parameter language model is going to be huge. You probably don't want the biggest model just for messing around.

Re: OPT: Open Pre-trained Transformer Language Models

#217

Post by moyapchen »

p1esk wrote: post #28

We are also releasing our logbook detailing the infrastructure challenges we faced Where’s the logbook?

And it’s live!

https://github.com/facebookresearch/metaseq

Logbook links in specific: https://github.com/facebookresearch/metaseq/blob/main/projec...

Re: OPT: Open Pre-trained Transformer Language Models

#218

Post by lumost »

teaearlgraycold wrote: post #148
lumost wrote: post #37

I often wonder if OpenAIs decision not to open gpt-3 was because it was to expensive to train relative to its real value. They’ve hidden the model behind an api where they can filter out most of the dumb behaviors, while everyone believes they are working on something entirely different.

> They’ve hidden the model behind an api where they can filter out most of the dumb behaviors What do you mean by this?

There is some evidence that the OpenAI GPT-3 APIs have a human in the loop for bad examples. They may also have a number of filters to exclude certain words/patterns/other rules.

The challenge with such rule and human in the loop systems is that the long-tail of these problems is huge, and fat. Meaning that you generally can't make a product which doesn't have full generalization. That it took ~1.5 years to open the GPT-3 API inclines me to think that they've run into similar problems. We're also not seeing the long pitched swarm of GPT enabled content despite the API being open for ~10 months.

Re: OPT: Open Pre-trained Transformer Language Models

#219

Post by dhzhzjsbevs »

andybak wrote: post #200
dhzhzjsbevs wrote: post #188

Earlier quoted context omitted.

Well, yea? You lot stopped caring about being seen long ago.

Both your comments indicate that you regard everyone here as some kind of homogonous group who share the same views - whilst you are somehow outside or different. That's a bit like sitting in a traffic jam complaining about the other cars. You are one of us and probably not a huge outlier either in most regard. I don't know why you have ended up with a me vs them perception but it's probably fairly unhealthy and I ho…

How did you get that from my comments?

Guy was clearly trying to setup a narrative.

Re: OPT: Open Pre-trained Transformer Language Models

#220

Post by adamsmith143 »

robbedpeter wrote: post #215
etaioinshrdlu wrote: post #35

What type of hardware would you need to run it?

A cluster of many $8000+ gpus. You're looking at around 350GB of vram, so 30 12gb gpus - a 3090 will cost around $1800, so $54k on the gpus, probably another $15k in power, cooling, and infrastructure, $5k in network, and probably another $20k in other costs to bootstrap it. Or wait 10 years, if gpu capacity scales with Moore's law, consumer hardware should be able to run a ~400GB model locally.

Almost no one does this on prem. What would this cost on AWS?
Post reply on HN