It's no good for training, only inference, but there's salad.com that has 10k's of DAU with GPUs and they've built a managed container service, with a inference API coming soon https://salad.com/salad-inference-endpoints
Ask HN: Can you crowdfund the compute for GPT?
111–120 of 157 posts
Re: Ask HN: Can you crowdfund the compute for GPT?
#112I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
I tried building a startup around this back in ~2012 called "Netkine". The basic premise was that compute is fungible and that all compute has a price (regardless of whether it is from a datacenter, or from someone's desktop PC). We seeded different compute on different networks around the SF Bay Area (either in low cost datacenters, or different networks like Comcast/AT&T) and had a customized version of Linux which could securely boot on a Windows gaming machine. You could then carve out different sized virtual machines on any machine in the network, and it would seamlessly create a VPN tying each of the instances together.
Our biggest problem was finding customers on the demand side. What is the killer application which can take use of that kind of compute with those kinds of networking properties? What are the use cases? No one has written software that takes advantage of that kind of compute, because it's essentially a solution looking for a problem.
Re: Ask HN: Can you crowdfund the compute for GPT?
#113Elutherai?
Re: Ask HN: Can you crowdfund the compute for GPT?
#114It's not that easy. Access to enough compute is one thing. However, you also need a proper dataset (beyond Common Crawl and Wikipedia), excellent research expertise and engineering capabilities. So even if you throw money or free credits for cloud compute out there it will not be enough. We've seen this happen with EleutherAI who were not capable of reaching their initial target of "replicating" GPT-3 and could only…
We solved the proper dataset part at least. https://arxiv.org/abs/2101.00027
My contribution was around 19,000 books.
Re: Ask HN: Can you crowdfund the compute for GPT?
#115Earlier quoted context omitted.
There is Open Street Map (or Wikipedia for that matter). A large enough army of volunteers could produce or tag a dataset that rivals Google's data, but it would be a lot of work.
> tag a dataset that rivals Google's data Yeah, try rival Google's Street View data.
Open Street Map is used by Amazon, Apple, Baidu Maps, Facebook and Microsoft. It might not be as good as Google, but it is decent.
And Google have no equivalent to Wikipedia.
Re: Ask HN: Can you crowdfund the compute for GPT?
#116I've often wondered why a service doesn't exist that allows you to rent out your graphics card for the large data processing needed for training models. Like mining bitcoin except you are doing something actually useful and getting paid actual money for it. Example: - Company Alpha needs $40,000,000 worth of cloud computing for their training model - Company Beta provides them said cloud computing for $30,000,000 fro…
It does exist! Those most notable are vast.ai and runpod.io
I've been able to run ML experiments that would have cost 10s of thousands of dollars for a tiny fraction of that using vast.
Re: Ask HN: Can you crowdfund the compute for GPT?
#117Maybe someone can make a cryptocurrency where mining and "proof of work" is accomplished by submitting an adjustment to weights and biases such that it improves the test score on a public dataset.
Re: Ask HN: Can you crowdfund the compute for GPT?
#118It's no different from any other type of server I think? Crowdfunding compute comes with a lot of security and privacy concerns and often if you're building a product putting that part of it in the hands of someone reliable is preferable. It depends on what part of what you're building is your core intellectual property and such. I'll ask back: what products would you build on top of a crowdfunded-compute GPT like mo…
Personally? One I'm interested in is a personal assistant that has access to 100+ APIs that I can command solely with text e.g ability to schedule future tasks, ability to get answers to questions around real world events and data available through APIs, ability to combine a lot of these things. What Alexa could/should have been. Separately, examination of religious and historic tests in alternate languages providing…
> Separately, examination of religious and historic tests in alternate languages providing superior transliteration and translation into english than what humans could achieve.
I'd be interested in something similar to this as well - but I think part of the problem is that depending on the language, the text might only be available as digital scans without OCR. In that case, given that it would be in difficult-to-OCR languages, how could it be fed to an AI model?
Re: Ask HN: Can you crowdfund the compute for GPT?
#119Earlier quoted context omitted.
Personally? One I'm interested in is a personal assistant that has access to 100+ APIs that I can command solely with text e.g ability to schedule future tasks, ability to get answers to questions around real world events and data available through APIs, ability to combine a lot of these things. What Alexa could/should have been. Separately, examination of religious and historic tests in alternate languages providing…
From my understanding that is exactly what Microsoft's XiaoIce is doing in the Chinese market. Just with a text interface instead of a voice interface. I guess there's not much interest to develop something equivalent for the English-speaking market because it's difficult to monetize. https://en.wikipedia.org/wiki/Xiaoice https://arxiv.org/pdf/1812.08989.pdf (actually a better introduction than the wikipedia)
The examples in the pdf are really advanced for something that was launched in 2014.
Re: Ask HN: Can you crowdfund the compute for GPT?
#120Yes, you can. It's not rocket science - lambdalabs and preemptible instances on various public clouds are possible choices.
And you probably should.