Because GPT models will be made smaller eventually.
Like how people mirror the initial Stable Diffusion model, only to find better, faster and smaller new versions later.
Ask HN: Can you crowdfund the compute for GPT?
51–60 of 157 posts
Re: Ask HN: Can you crowdfund the compute for GPT?
#52Distributed training could be done but how would data validation work? There would need to be some kind of voting per item as you need to assume adversarial input ( honestly a crypto token could work here for governance ). EleutherAI could be used to bootstrap but the hardware reqs for Neo-X are quite steep ( around 20 GB cuda memory per card , so you could only allow those who had a 3090+ and are skilled enough to install cuda 11 and do devops ). Im happy to contribute though!
Re: Ask HN: Can you crowdfund the compute for GPT?
#53I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
There is this https://vast.ai/hosting
Also https://www.runpod.io/ (which I’ve been a very happy user of)
Re: Ask HN: Can you crowdfund the compute for GPT?
#54[flagged]
Re: Ask HN: Can you crowdfund the compute for GPT?
#55I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
>Like mining Bitcoin except you are doing something actually useful and getting paid actual money for it
A business paying USD is never going to be competitive with a decentralized crypto compute market. Better to build something just like mining BTC, except you do useful work, and are paid in a cryptocurrency you can exchange for USD. Then businesses can build on top of that to make it more user friendly.
The Golem network already lets you do this with CPU. It's way cheaper than centralized cloud computing could ever be, less than $0.003 per core per hour.
There are plans to add GPU support to Golem, but they have been on the backlog for a long time because there is a shortage of devs.
Re: Ask HN: Can you crowdfund the compute for GPT?
#56I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
Vast.ai does this
Re: Ask HN: Can you crowdfund the compute for GPT?
#57I believe distributed compute *could* be one of the few applications where crypto might make sense? You could earn tokens by supplying compute and pay to get it, and the token price is a natural market price for compute.
Re: Ask HN: Can you crowdfund the compute for GPT?
#58Check out https://petals.ml/ .
Chat demo here: http://chat.petals.ml/
Seems to work well, albeit very slowly…
Re: Ask HN: Can you crowdfund the compute for GPT?
#59It's no different from any other type of server I think? Crowdfunding compute comes with a lot of security and privacy concerns and often if you're building a product putting that part of it in the hands of someone reliable is preferable. It depends on what part of what you're building is your core intellectual property and such. I'll ask back: what products would you build on top of a crowdfunded-compute GPT like mo…
Personally? One I'm interested in is a personal assistant that has access to 100+ APIs that I can command solely with text e.g ability to schedule future tasks, ability to get answers to questions around real world events and data available through APIs, ability to combine a lot of these things. What Alexa could/should have been. Separately, examination of religious and historic tests in alternate languages providing…
From my understanding that is exactly what Microsoft's XiaoIce is doing in the Chinese market. Just with a text interface instead of a voice interface.
I guess there's not much interest to develop something equivalent for the English-speaking market because it's difficult to monetize.
https://en.wikipedia.org/wiki/Xiaoice
https://arxiv.org/pdf/1812.08989.pdf (actually a better introduction than the wikipedia)
Re: Ask HN: Can you crowdfund the compute for GPT?
#60Earlier quoted context omitted.
I think the second part of what you said is key. Seeing this thread hit top of HN I can see there's a desire or need for something like this, but how you'd even go implementing it, I have no idea. I have mildly speculative ideas but nothing that would result in a successful outcome. So it's like OK, lets say you crowdsource even $20m, now what. What's the structure of this thing that has $20m, how do you use it, what…
You'd need a good volunteer team of experts willing to shepherd it first and foremost. For it to be a volunteer team you'd need everyone to ideologically believe in the idea. They should be able to build an outline, line item wise, how each dollar is spent. I think that will help get it started. Then swap to a non-profit monthly sub model where people are simply paying for use (pay out the cost for labor and hardware…
I like the analogy of a utility bill, that makes sense.