Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-Neo – Building a GPT-3-sized model, open source and free

eleuther.ai

Post reply on HN
Print view View original

211–220 of 260 posts

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#211

Post by hooande »

napier wrote: post #129
hooande wrote: post #66

In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…

>"what would you do with a team of hundreds of people who can solve arbitrary problems only by googling them?" What would you do with a team of hundreds of people who can instantly access an archive comprising the sum total of digitized human knowledge and use it to solve problems?

We have that now, it's called googling. you could easily hire 100 people to do that job, but you'd have to pay them at least $15/hr now on the US. Say equivalent gpt-3 servers cost a fraction of that. How do you make money with that resource?

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#212

Post by sailingparrot »

mortehu wrote: post #203
kordlessagain wrote: post #191

Earlier quoted context omitted.

The model is huge and is currently run in the cloud on many machines.

It's only 175 billion parameters, so presumably it can fit on a single computer with 1024 GB RAM.

On CPU the latency would be absolutly prohibitive to the point of being useless.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#213

Post by sailingparrot »

Mizza wrote: post #123

Serious question: is there a warez scene for trained models yet? (I don't know how the model is accessed - are users of mainline GPT-3 given a .pb and a stack of NDAs, or do they have to access it through access-controlled API?) Wherever data is desired by many but held by a few, a pirate crew inevitably emerges.

Checkpoint is not shared with customers, you only get access to an API endpoint.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#214

Post by stingraycharles »

mortehu wrote: post #203
kordlessagain wrote: post #191

Earlier quoted context omitted.

The model is huge and is currently run in the cloud on many machines.

It's only 175 billion parameters, so presumably it can fit on a single computer with 1024 GB RAM.

Wouldn't you need this model to be in GPU RAM instead of regular RAM, though?

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#215

Post by sailingparrot »

mirekrusin wrote: post #108

I'd gladly contribute (power and) few of idle GTX cards I have to public peer/volunteer/seti@home-like project if result snapshot(s) are available publicly/to registered, active contributors.

SETI@home style distributed computation is not suitable for training something like GPT-3, unlike for SETI, the unit of work a node can do before needing to share it's output with the next node is really small, so very fast interconnect between the nodes is needed (Infiniband and NVLink is used in clusters to train it). It would probably take a decade to train such a model over regular internet.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#216

Post by typon »

sailingparrot wrote: post #212
mortehu wrote: post #203

Earlier quoted context omitted.

It's only 175 billion parameters, so presumably it can fit on a single computer with 1024 GB RAM.

On CPU the latency would be absolutly prohibitive to the point of being useless.

For training yes, but not for inference.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#219

Post by sailingparrot »

typon wrote: post #216
sailingparrot wrote: post #212

Earlier quoted context omitted.

On CPU the latency would be absolutly prohibitive to the point of being useless.

For training yes, but not for inference.

Inference on GPU is already very slow on the full-scale non-distilled model (in the 1-2 sec range iirc), on CPU it would be an order of magnitude more.

Re: GPT-Neo – Building a GPT-3-sized model, open source and free

#220

Post by glebshevchuk »

devonkim wrote: post #167
hooande wrote: post #66

In my experience, the output from GPT-3, DALL-E, et al is similar to what you get from googling the prompt and stitching together snippets from the top results. These transformers are trained on "what was visible to google", which provides the limitation on their utility. I think of the value proposition of GPT-X as "what would you do with a team of hundreds of people who can solve arbitrary problems only by googling…

The current problem is that we don’t have a reliable, scalable way to merge in features of knowledge engines that have ontological relationships of entities with generative engines that are good for making more natural looking or sounding qualitative output. There’s certainly research going on to join them together but it’s just not getting the kind of press releases as the generative and pattern recognition stuff th…

Agreed, but that's because they're hard to integrate together: one is concerned with enumerating over all facts that humans know about (a la Cyc) and the other is concerned with learning those directly from data. Developing feedback systems that combine these two would be quite exciting.
Post reply on HN