I created a subReddit for this in case enough people want to actually try to make this happen.
Ask HN: Can you crowdfund the compute for GPT?
71–80 of 157 posts
Re: Ask HN: Can you crowdfund the compute for GPT?
#72I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
There's a lot of problems. 1. How can I confirm that you've done the computation? 2. Privacy and security issues. Can I trust you too process my sensitive information? 3. Availability: is there a guarranty you won't just do half of it and then be on a hiatus for 2 months. But for everywhere these problems are solved we have decentralized cloud computing. For others you need to solve these problems.
There are a number of entities working on this problem. Plenty of papers in Arvix on it as well.
Re: Ask HN: Can you crowdfund the compute for GPT?
#73Earlier quoted context omitted.
There's a lot of problems. 1. How can I confirm that you've done the computation? 2. Privacy and security issues. Can I trust you too process my sensitive information? 3. Availability: is there a guarranty you won't just do half of it and then be on a hiatus for 2 months. But for everywhere these problems are solved we have decentralized cloud computing. For others you need to solve these problems.
Some blockchain projects attempted to tackle this. Check out Akash.
Akash is more about k8s, paid for by tokens.
Re: Ask HN: Can you crowdfund the compute for GPT?
#74I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
Why not just use AWS?
Price and dependency on a single entity.
Re: Ask HN: Can you crowdfund the compute for GPT?
#75I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
There's a lot of problems. 1. How can I confirm that you've done the computation? 2. Privacy and security issues. Can I trust you too process my sensitive information? 3. Availability: is there a guarranty you won't just do half of it and then be on a hiatus for 2 months. But for everywhere these problems are solved we have decentralized cloud computing. For others you need to solve these problems.
The Lightning Network may be used for invoices, identity and payment of use of model. It can also likely be used to handle prompts. Regarding privacy, not all things need to be private and if they do they should be run on a single tenant infrastructure. OpenAI is a multi-tenant cloud model, so guarantee of privacy there may be protected only by license and use agreement.
Re: Ask HN: Can you crowdfund the compute for GPT?
#76I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
The compute workloads for AI/ML are vastly different than mining, which means that the hardware requirements are totally different (and actually a huge capex investment).
Individuals renting out their home compute for things like this suffer from the problem of supporting so many different types of hardware profiles. BOINC is a prime example of this, but it doesn't have a funding mechanism.
Re: Ask HN: Can you crowdfund the compute for GPT?
#77But what about something for individuals? You contribute n GPU minutes to the community adhoc pool, you get to use n minutes in parallel from said pool. Some proof of work underlying blockchain. Just for individuals, no promises of deep data / model privacy. Has anyone implemented something like this?
Check out Akash Network https://akash.network/
Support for GPUs are still in the works.
The larger issue here though is the amount of bandwidth and memory required. Downloading billions of images to train a model just isn't going to work.
People are working on splitting the training into smaller chunks (together.xyz is one example), but they aren't quite there yet.
Re: Ask HN: Can you crowdfund the compute for GPT?
#78Earlier quoted context omitted.
Surprised no one has commented this but the latency requires the model to be trained in tiny fragments on each device which is currently a field of research that is being explored. As it stands now basically all of a model needs to be loaded into memory. There’s a whole field here and people exploring this problem, colloquially solving this would enable Federated Learning and whoever figures this out will far eclipse…
Where can I read more about this field of research?
https://arxiv.org/ has a ton of papers on it.
Re: Ask HN: Can you crowdfund the compute for GPT?
#79I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
Surprised no one has commented this but the latency requires the model to be trained in tiny fragments on each device which is currently a field of research that is being explored. As it stands now basically all of a model needs to be loaded into memory. There’s a whole field here and people exploring this problem, colloquially solving this would enable Federated Learning and whoever figures this out will far eclipse…
Well, that explains the lack of a BOINC client for ML tasks at the moment.
Re: Ask HN: Can you crowdfund the compute for GPT?
#80I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
Company Alpha will probably just call up AWS or Azure and sleep easy. Company Beta can't really compete because they have to pay both themselves and their contributors; AWS could sell ML compute at-cost and still turn a profit. What made this type of business feasible/attractive for cryptocurrency is that the miners were Company Beta. There wasn't two mouths to feed or two pockets to line, just a direct and transpare…
It would require a problem that reasonably fits the solution.
You for example get a keyboards and a set of hands with each node, a webcam, a microphone.
I have no real idea but it seems the hand of cards isn't hopeless.