Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Could you train a ChatGPT-beating model for $85k and run it in a browser?

simonwillison.net

Post reply on HN
Print view View original

51–60 of 175 posts

Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?

#51

Post by ftxbro »

robertlagrant wrote: post #35
icelancer wrote: post #25

Earlier quoted context omitted.

Yeah, the constant barrage of "THIS IS AS GOOD AS CHATGPT AND IS PRIVATE" screeds from LLaMA-based marketing projects are getting ridiculous. They're not even remotely close to the same quality. And why would they be? I want the best LLMs to be open source too, but I'm not delusional enough to make insane claims like the hundreds of GitHub forks out there.

> I want the best LLMs to be open source too How do you do this without being incredibly wealthy?

Yes a rugged individual would have to be incredibly wealthy to do it!

But maybe the governments will make one and maintain it with taxes as an infrastructure service, like roads, giving everyone expanded powers of cognition, memory, and expertise, and raising the consciousnesses of humanity to new heights. Probably in USA it wouldn't happen if we judge ourselves only in zero sum relation to others - helping everyone would be a wash and only waste our money!

Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?

#52

Post by ftxbro »

whalesalad wrote: post #49

Are there any training/ownership models like Folding@Home? People could donate idle GPU resources in exchange for access to the data, and perhaps ownership. Then instead of someone needing to pony up $85k to train a model, a thousand people can train a fraction of the model on their consumer GPU and pool the results, reap the collective rewards.

Yes there is petals/bloom https://github.com/bigscience-workshop/petals but it's not so great. Maybe it will improve or a better one will come.

Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?

#53

Post by nickthegreek »

robertlagrant wrote: post #35
icelancer wrote: post #25

Earlier quoted context omitted.

Yeah, the constant barrage of "THIS IS AS GOOD AS CHATGPT AND IS PRIVATE" screeds from LLaMA-based marketing projects are getting ridiculous. They're not even remotely close to the same quality. And why would they be? I want the best LLMs to be open source too, but I'm not delusional enough to make insane claims like the hundreds of GitHub forks out there.

> I want the best LLMs to be open source too How do you do this without being incredibly wealthy?

crowd source to pay for the gpu rentals.

Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?

#54

Post by whalesalad »

ftxbro wrote: post #52
whalesalad wrote: post #49

Are there any training/ownership models like Folding@Home? People could donate idle GPU resources in exchange for access to the data, and perhaps ownership. Then instead of someone needing to pony up $85k to train a model, a thousand people can train a fraction of the model on their consumer GPU and pool the results, reap the collective rewards.

Yes there is petals/bloom https://github.com/bigscience-workshop/petals but it's not so great. Maybe it will improve or a better one will come.

Really interesting live monitor of the network: http://health.petals.ml

Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?

#55

Post by polishdude20 »

ftxbro wrote: post #52
whalesalad wrote: post #49

Are there any training/ownership models like Folding@Home? People could donate idle GPU resources in exchange for access to the data, and perhaps ownership. Then instead of someone needing to pony up $85k to train a model, a thousand people can train a fraction of the model on their consumer GPU and pool the results, reap the collective rewards.

Yes there is petals/bloom https://github.com/bigscience-workshop/petals but it's not so great. Maybe it will improve or a better one will come.

I wonder how they handle illegal content. Like, if you're running training data on your computer, what's to stop someone else's data that is illegal, from being uploaded to your computer as part of training?

Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?

#56

Post by pessimizer »

dekhn wrote: post #20
version_five wrote: post #16

If you have ~100k to spend, aren't there options to buy a gpu rather than just blow it all on cloud? How much is an 8xA100 machine? 4xA100 is 75k, 8 is 140k https://shop.lambdalabs.com/deep-learning/servers/hyperplane...

you're comparing the capital cost of acquiring a GPU machine with the operational cost of renting one in the cloud. Ignoring the operational costs of on-prem hardware is pretty common, but those costs are significant and can greatly change the calculation.

Or from another perspective, comparing the cost of training one model in the cloud to the cost of training as many as you want on your machine, then (as mentioned by siblings) selling the machine for nearly as much as you paid for it, unless there's some shortage, in which case you'll get more back than you paid for it.

One is buying capital that produces models, the other is buying a single model.

Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?

#57

Post by throwaway50601 »

capableweb wrote: post #45
dekhn wrote: post #20

Earlier quoted context omitted.

you're comparing the capital cost of acquiring a GPU machine with the operational cost of renting one in the cloud. Ignoring the operational costs of on-prem hardware is pretty common, but those costs are significant and can greatly change the calculation.

Heh, you work at AWS or Google Cloud perhaps? ;) (Only joking about this as I constantly see employees from AWS/GCloud and other cloud providers claim that cloud is always cheaper than hosting things yourself) Sure, if you're planning to service a large number of users, building your infrastructure in-house might be a bit overkill, as you'll need a infrastructure team to service it as well. If you're just want to buy…

Cloud gives you very good price for what they offer - excellent reliability, hyper-scalability. Most people don't need either and use it as a glorified VPS host.

Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?

#60

Post by ellisv »

whalesalad wrote: post #49

Are there any training/ownership models like Folding@Home? People could donate idle GPU resources in exchange for access to the data, and perhaps ownership. Then instead of someone needing to pony up $85k to train a model, a thousand people can train a fraction of the model on their consumer GPU and pool the results, reap the collective rewards.

That’d be cool but I don’t think most idle consumer GPUs (6-8GB) would have large enough memory for a single iteration (batch size 1) of modern LLMs.

But I’d love to see more federated/distributed learning platforms.

Post reply on HN