Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can you crowdfund the compute for GPT?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 157 posts

Re: Ask HN: Can you crowdfund the compute for GPT?

#121

Post by sinenomine »

beernet wrote: post #12

It's not that easy. Access to enough compute is one thing. However, you also need a proper dataset (beyond Common Crawl and Wikipedia), excellent research expertise and engineering capabilities. So even if you throw money or free credits for cloud compute out there it will not be enough. We've seen this happen with EleutherAI who were not capable of reaching their initial target of "replicating" GPT-3 and could only…

Data isn't the hard part here, plenty is available, even with all the necessary preprocessing.

Re: Ask HN: Can you crowdfund the compute for GPT?

#122

Post by asim »

generalizations wrote: post #118
asim wrote: post #19

Earlier quoted context omitted.

Personally? One I'm interested in is a personal assistant that has access to 100+ APIs that I can command solely with text e.g ability to schedule future tasks, ability to get answers to questions around real world events and data available through APIs, ability to combine a lot of these things. What Alexa could/should have been. Separately, examination of religious and historic tests in alternate languages providing…

> Separately, examination of religious and historic tests in alternate languages providing superior transliteration and translation into english than what humans could achieve. I'd be interested in something similar to this as well - but I think part of the problem is that depending on the language, the text might only be available as digital scans without OCR. In that case, given that it would be in difficult-to-OCR…

Write a model to do image to text? In the case of things like the Quran pretty much all of it is digitized already so consumable by a model. Other things could be figured out over time. I think for a lot of things this could lead to huge breakthroughs if done right.

Re: Ask HN: Can you crowdfund the compute for GPT?

#123

Post by bobkazamakis »

alasdair_ wrote: post #106
rysertio wrote: post #65

Earlier quoted context omitted.

There's a lot of problems. 1. How can I confirm that you've done the computation? 2. Privacy and security issues. Can I trust you too process my sensitive information? 3. Availability: is there a guarranty you won't just do half of it and then be on a hiatus for 2 months. But for everywhere these problems are solved we have decentralized cloud computing. For others you need to solve these problems.

>1. How can I confirm that you've done the computation? The same problem applies to things like Mechanical Turk and other croudsourcing. The way I've dealt with the issue in the past is to start with zero trust and to have them do computations that I already know the answer to. After that, they do computations that are matched with a random other participant (the two results should match, if they don't, compare again…

> have them do computations that I already know the answer to.

By definition, this is not having them do any computation. The proper solution at this time would be some trapdoor function that is easily verifiable (proof of work), at least while P != NP

Re: Ask HN: Can you crowdfund the compute for GPT?

#124

Post by mitchellpkt »

rysertio wrote: post #65
alexb_ wrote: post #25

I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…

There's a lot of problems. 1. How can I confirm that you've done the computation? 2. Privacy and security issues. Can I trust you too process my sensitive information? 3. Availability: is there a guarranty you won't just do half of it and then be on a hiatus for 2 months. But for everywhere these problems are solved we have decentralized cloud computing. For others you need to solve these problems.

Good questions. Hmm do you think maybe it would be viable if the payouts were based on [model] performance rather than ostensible training time?

There's a useful asymmetry we can exploit: finding weights that perform well is computationally intensive and takes time, but scoring a set of weights is fast and easy.

A number cruncher could spend 2 weeks training a model, and then when they submit the results it takes me 10 seconds to score the model - to verify the quality of the results, and calculate the performance-based payout. In the #1 or #3 scenario where they didn't do or didn't complete the computations, they wouldn't have a well-trained model to submit for payout. (The lost time in #3 is inconvenient in time-sensitive situations, but mechanisms exist to address that - SLAs, up-front collateral, etc)

Regarding privacy, that's an EXTREMELY good and important question. There's some really neat prior art for privacy-preserving machine learning that could be useful here, e.g. https://arxiv.org/abs/2106.07229 "Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Network"

(note I'm approaching this as an interesting DistML thought experiment, not proposing it as an immediately viable or sensible initiative)

Re: Ask HN: Can you crowdfund the compute for GPT?

#125

Post by hooande »

sillysaurusx wrote: post #114
beernet wrote: post #12

It's not that easy. Access to enough compute is one thing. However, you also need a proper dataset (beyond Common Crawl and Wikipedia), excellent research expertise and engineering capabilities. So even if you throw money or free credits for cloud compute out there it will not be enough. We've seen this happen with EleutherAI who were not capable of reaching their initial target of "replicating" GPT-3 and could only…

We solved the proper dataset part at least. https://arxiv.org/abs/2101.00027 My contribution was around 19,000 books.

isn't Common Crawl much, much larger than this? ~6 pebibytes from what I remember

Re: Ask HN: Can you crowdfund the compute for GPT?

#126

Post by vagabund »

elicksaur wrote: post #87
latchkey wrote: post #78

Earlier quoted context omitted.

https://arxiv.org/ has a ton of papers on it.

Do you have some good search terms to get started down the rabbit hole?

Probably the biggest recent result: https://arxiv.org/abs/2209.04836 (author thread: https://twitter.com/SamuelAinsworth/status/15697194946455265...)

See also: https://github.com/learning-at-home/hivemind

and more to OP's incentive structure: https://docs.bittensor.com/

Latter two intend to beat latency with Mixture-of-Expert models (MoEs). If the results of the former hold, it shows that with a simple algorithmic transformation you can merge two independently trained models in weight-space and have performance functionally equivalent to a model trained monolithically.

Re: Ask HN: Can you crowdfund the compute for GPT?

#129

Post by drakenot »

brap wrote: post #70
IshKebab wrote: post #35

Earlier quoted context omitted.

Yeah someone posted something like that on HN maybe a year ago. It's not especially useful though because most companies (who are actually going to pay for this service) aren't going to want to send their training data to random people, and ML training needs high performance links between the cards. Plus you'd have to deal with the fact that you're running on 100 different GPU models.

In addition to privacy, performance, and portability, also: * Servers in a datacenter are much more reliable than a network of PCs (power goes off, someone decides to play Crysis, etc) * People will find ways to scam you (pretend like they’re doing the calculation while not actually doing it) * Economies of scale means a datacenter will probably be cheaper than what you’d have to pay the PC owners (power consumption,…

What if you remove the financial incentive?

I'd contribute my GPU time to a Folding@Home style project if it meant that we had powerful, open LLMs that were free to use. I'm positive many others would as well.

As far as worrying about scammers, could you send the same compute task and training data to multiple clients and validate the results against each other? If they differed, you could throw the results and try again, or send it to a 3rd to break the tie.

Re: Ask HN: Can you crowdfund the compute for GPT?

#130

Post by rubyist5eva »

throwaway14356 wrote: post #83
rubyist5eva wrote: post #36

Earlier quoted context omitted.

> Like mining bitcoin except you are doing something actually useful and getting paid actual money for it But then how would crypto-scammers run ponzi after ponzi after ponzi if they did that?

oh? That seems to be the answer. They wont actually have to care what the compute costs if they can print money back stage.

Sounds like work, as long as they prove they can prove they did it somehow - perhaps all the other computers that are sharing compute could also verify they did it somehow?
Post reply on HN