Now imagine loading 3.9 GB each time you want to interact with a webpage
Yeah, I’ve used Jira.
Could you train a ChatGPT-beating model for $85k and run it in a browser?
21–30 of 175 posts
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#22If you have ~100k to spend, aren't there options to buy a gpu rather than just blow it all on cloud? How much is an 8xA100 machine? 4xA100 is 75k, 8 is 140k https://shop.lambdalabs.com/deep-learning/servers/hyperplane...
You can sell the A100 after once you're done as well. Possibly even at profit?
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#23If you have ~100k to spend, aren't there options to buy a gpu rather than just blow it all on cloud? How much is an 8xA100 machine? 4xA100 is 75k, 8 is 140k https://shop.lambdalabs.com/deep-learning/servers/hyperplane...
you're comparing the capital cost of acquiring a GPU machine with the operational cost of renting one in the cloud. Ignoring the operational costs of on-prem hardware is pretty common, but those costs are significant and can greatly change the calculation.
Remember to discount the tax depreciation for the hardware and deduct any potential future gains from either reselling it or using it.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#24So cool it runs on a browser /sarcasm/ i might not even need a computer. Or internet when we are at it. It either runs locally or it runs on the cloud. Data could come from both locations as well. So it's mostly technically irrelevant if it's displaying in a browser or not. Except when it comes to usability. I don't get it why people love software running in a browser. I often close important tools i have not saved w…
> I don't get it why people love software running in a browser. If you read the article, part of the argument was for the sandboxing that the browser provides. "Obviously if you’re going to give a language model the ability to execute API calls and evaluate code you need to do it in a safe environment! Like for example... a web browser, which runs code from untrusted sources as a matter of habit and has the most thor…
OSX does app sandboxing as well (not everywhere). But yeah, you're right i only skimmed the content and missed that part.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#25His estimate is that you could train a LLaMA-7B scale model for around $82,432 and then fine-tune it for a total of less than $85K. But when I saw the fine tuned LLaMA-like models they were worse in my opinion even than GPT-3. They were like GPT-2.5 or like that. Not nearly as good as ChatGPT 3.5 and certainly not ChatGPT-beating. Of course, far enough in the future you could certainly run one in the browser for $85K…
Yeah, the constant barrage of "THIS IS AS GOOD AS CHATGPT AND IS PRIVATE" screeds from LLaMA-based marketing projects are getting ridiculous. They're not even remotely close to the same quality. And why would they be?
I want the best LLMs to be open source too, but I'm not delusional enough to make insane claims like the hundreds of GitHub forks out there.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#26A wonderful thing about software development is that there is so much reserved space for creativity that we have huge gaps between costs and value. Whether the average person could do this for 85k I'm uncertain of - but there is a very significant slice of people that could do it for well under 85k now that the ground work has been done. This leads to the hilarious paradox where a software based business worth millio…
Nit: not to write but to run. The cost of development is not considered in these calculations.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#27Alpaca uses knowledge distillation (it's trained on outputs from OpenAI models).
It's something to keep in mind. You're teaching your model to copy an other model's outputs.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#28So cool it runs on a browser /sarcasm/ i might not even need a computer. Or internet when we are at it. It either runs locally or it runs on the cloud. Data could come from both locations as well. So it's mostly technically irrelevant if it's displaying in a browser or not. Except when it comes to usability. I don't get it why people love software running in a browser. I often close important tools i have not saved w…
Broswer software is great because I don't have to build separate versions for Windows, Mac, and Linux, or deal with app stores, or figure out how to update old versions.
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#29If you have ~100k to spend, aren't there options to buy a gpu rather than just blow it all on cloud? How much is an 8xA100 machine? 4xA100 is 75k, 8 is 140k https://shop.lambdalabs.com/deep-learning/servers/hyperplane...
you're comparing the capital cost of acquiring a GPU machine with the operational cost of renting one in the cloud. Ignoring the operational costs of on-prem hardware is pretty common, but those costs are significant and can greatly change the calculation.
For a server farm, sure, for one machine, I don't know. Assuming it plugs into a normal 15A circuit, and you have a we-work or something where you don't pay for power, is the operational cost of one machine really material?
Re: Could you train a ChatGPT-beating model for $85k and run it in a browser?
#30Keep in mind that image transformer models like stable diffusion are generally smaller than language models, so they are easier to fit in wasm space. Also. you can finetune llama-7b on a 3090 for about $3 using LoRA.
Only for images. People want to generate videos next and those models will be likely GPT-sized.
There is a video model making the rounds on /r/stablediffusion and it is just a tiny bit larger than Stable Diffusion.