Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can you crowdfund the compute for GPT?

news.ycombinator.com

Post reply on HN
Print view View original

101–110 of 157 posts

Re: Ask HN: Can you crowdfund the compute for GPT?

#101

Post by eternalban »

k2xl wrote: post #66
cuuupid wrote: post #39

Earlier quoted context omitted.

Surprised no one has commented this but the latency requires the model to be trained in tiny fragments on each device which is currently a field of research that is being explored. As it stands now basically all of a model needs to be loaded into memory. There’s a whole field here and people exploring this problem, colloquially solving this would enable Federated Learning and whoever figures this out will far eclipse…

Where can I read more about this field of research?

A broad covering term is MLOps (like DevOps).

https://en.wikipedia.org/wiki/MLOps

Armed with that term, we get (haven’t read):

Machine Learning Operations (MLOps): Overview, Definition, and Architecture

https://arxiv.org/abs/2205.02302

[+ps]

Better resource: https://ml-ops.org/

Based on my (limited) exposure to date, there is tremendous opportunity for software engineers and architects to make impact in ML systems. There is a pronounced lack of seasoned engineering talent (outside of big players like DataBricks, et al) and this knowledge gap sits behind an experience curve that mere IQ can’t jump over. Our experience as software architects and engineers is very valuable.

Know this and recognize the value you will bring to the table.

Re: Ask HN: Can you crowdfund the compute for GPT?

#102

Post by NonTrivialZer0 »

alexb_ wrote: post #25

I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…

The ACT token is a decentralized, token-based solution for powering AI inference that can be used to purchase compute resources on GCP, AWS, Azure, and the decentralized network. Each token is equivalent to a certain number of inferences by the AI Compute Unit (ACU), which is a stable average of GPU compute instances available in the pool.

Re: Ask HN: Can you crowdfund the compute for GPT?

#103

Post by NonTrivialZer0 »

smnrg wrote: post #10

Side hot take: 1. “AI” requires compute time for training (GPT, etc.) 2. If you use any “AI” service in the future, you could have to share idle computing power to improve that “AI.” 3. This may be tokenized via crypto. More contributions = more “AI” usage available for you. Could the use of tokens or cryptocurrency incentivize participants to contribute their idle computing power to facilitate the process beyond tho…

ACT includes a proof-of-compute mechanism to ensure the reliability of the resources on the decentralized network. Users can register their instances with the system and be rewarded with ACT tokens for doing so. The system will also continuously monitor the instances, and remove instances that fail to meet quality standards or provide inaccurate results. ACT also includes a reputation system that allows holders to track the performance of the resources on the decentralized network. Users that provide accurate and stable resources will have a higher reputation score and be more likely to be selected for providing compute resources. The decentralized network is managed by a process for adding and removing resources. This ensures that the network is always providing high-quality resources and maintain the stability of the ACU value.

https://docs.google.com/document/d/1NTsnUVRzBfK6y7WNX2JURogD...

Re: Ask HN: Can you crowdfund the compute for GPT?

#104

Post by waffletower »

Unfortunately, it can be difficult to crowd-source funding when funding services, like Kickstarter cave to opposition to controversial compute projects. Unstable Diffusion, despite quickly reaching their funding goals, was suspended on Kickstarter. https://www.kickstarter.com/projects/unstablediffusion/unsta...

Re: Ask HN: Can you crowdfund the compute for GPT?

#105

Post by Aromasin »

cuuupid wrote: post #39
alexb_ wrote: post #25

I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…

Surprised no one has commented this but the latency requires the model to be trained in tiny fragments on each device which is currently a field of research that is being explored. As it stands now basically all of a model needs to be loaded into memory. There’s a whole field here and people exploring this problem, colloquially solving this would enable Federated Learning and whoever figures this out will far eclipse…

To piggy back this comment, Federated Learning actually has a lot of other uses beyond just crowdsourcing. It has benefits to data privacy also. The medical industry has struggled sharing their data with other institutions without compromising their information. They're using it to decentralize the model making process.

Intel is doing some work with Penn on the subject now, if people want to read further: https://www.intel.com/content/www/us/en/newsroom/news/intel-...

Re: Ask HN: Can you crowdfund the compute for GPT?

#106

Post by alasdair_ »

rysertio wrote: post #65
alexb_ wrote: post #25

I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…

There's a lot of problems. 1. How can I confirm that you've done the computation? 2. Privacy and security issues. Can I trust you too process my sensitive information? 3. Availability: is there a guarranty you won't just do half of it and then be on a hiatus for 2 months. But for everywhere these problems are solved we have decentralized cloud computing. For others you need to solve these problems.

>1. How can I confirm that you've done the computation?

The same problem applies to things like Mechanical Turk and other croudsourcing. The way I've dealt with the issue in the past is to start with zero trust and to have them do computations that I already know the answer to. After that, they do computations that are matched with a random other participant (the two results should match, if they don't, compare against a third random participant).

Later, when trust has been developed, you can start assuming their work work is trustworthy, but still check it randomly with computations you know the answer to, or a second person doing the same computation.

Yes, this adds overhead (roughly 10% in aggregate) but it works fairly well. It works even better if there are penalties that you can impose for ffraudulent results (like cancelling ALL owed payouts).

Re: Ask HN: Can you crowdfund the compute for GPT?

#110

Post by alasdair_ »

moneywoes wrote: post #67
alexb_ wrote: post #25

I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…

Why not just use AWS?

> Why not just use AWS?

Physical goods vendors that sell through Amazon have found that once they sell something highly in demand, and profitable, Amazon uses all of their internal knowledge of the sales to create their own Amazon Basics version of the same good and then promotes that as a cheaper alternative, capturing most of the profit.

There is absolutely no reason not to believe that AWS will do the same thing and use their knowledge of the workloads and exact hardware requested to compete with you. The fact that they haven't done so yet isn't much evidence against them doing this in the future, since other parts of the same company already follow this approach.

Post reply on HN