The compute cost to train Stable Diffusion was $600k
1–9 of 9 posts
Re: The compute cost to train Stable Diffusion was $600k
#2> at market price $600k
I don’t think they are really spending $600k. Chances are that they own a GPU cluster so they don’t need to pay the cloud premium.
Re: The compute cost to train Stable Diffusion was $600k
#3Could training be crowd sourced among consumer GPUs like Folding at Home?
Re: The compute cost to train Stable Diffusion was $600k
#4Could training be crowd sourced among consumer GPUs like Folding at Home?
Probably not for a few years, you need a (maybe few) A100(s) to be able to backprop a model that big with float32.
Re: The compute cost to train Stable Diffusion was $600k
#5Is that relatively cheap or expensive?
I have no context for this, do other model dveelopers publish how much it costed to train them?
Re: The compute cost to train Stable Diffusion was $600k
#6> at market price $600k I don’t think they are really spending $600k. Chances are that they own a GPU cluster so they don’t need to pay the cloud premium.
Spot instance pricing for p4d.large is about a third the on-demand cost, though it's not clear what they're using to calculate market rate. An A100 card is roughly $10k retail, so it's not like "owning a GPU cloud" is that cheap a proposition either.
Re: The compute cost to train Stable Diffusion was $600k
#7Re: The compute cost to train Stable Diffusion was $600k
#8If you want to learn how and why stable diffusion works: https://www.paepper.com/blog/posts/how-and-why-stable-diffus...
Re: The compute cost to train Stable Diffusion was $600k
#9Is that relatively cheap or expensive? I have no context for this, do other model dveelopers publish how much it costed to train them?
Given the impact the model might have on the world, and that $600k is roughly equal to the annual salary of 1~3 top notch engineers... it's probably intended to mean relatively cheap.
For similar context, millions of VC money have been thrown around for much less impressive results.
#firstworldproblems