31–40 of 125 posts
Earlier quoted context omitted.
Well during training all GPUs were consuming max ~ 450W
Thanks, good to know. Perhaps it is different for diffusion; with llms, layers are generally split across gpus, meaning inference has to happen on one gpu before the values can be passed between the layer split.
All you need is a 4x 4090 GPUs and a dedicated 30 amp circuit.
You sure? In my experiments with multi gpu inference, I couldn't get anywhere close to max theoretical power draw.
His power supplies are 2x1500 Watt. That puts it at 3KW max which is more than a 20A circuit can provide (2400W).
The standard outlet is typically rated at 15 amps or 1800W. And the 15A breaker is on one circuit. You can get 20A circuits but they need to be wired for it, and replacing the breaker won't cut it.
Assuming his GPU is ~450W (his number) and power supplies are 80% efficient, well that means he's pulling close to ~2400 watts which is super close to the limit of a 20A circuit.
4 * 450 / 0.80 efficiency = 2250W.
That doesn't include the power consumed by the CPU or mother board or other things on that circuit. But a 170W CPU would easily push this over 2400W provided by a 20A circuit.
All you need is a 4x 4090 GPUs and a dedicated 30 amp circuit.
All you need is a 4x 4090 GPUs and a dedicated 30 amp circuit.
Commercial setups are not appropriate for typical 15 amp circuit loads.
All you need is a 4x 4090 GPUs and a dedicated 30 amp circuit.
Why are people downvoting this? Yes, you really do need a dedicated circuit to run this type of machine. You will trip your circuit breaker if you don't have sufficient wattage on the line to run something rated for this power draw. Commercial setups are not appropriate for typical 15 amp circuit loads.
All you need is 4x 4090 GPUs to Train Your Own Model -- and $12000 to buy them
Nvidia stopped manufacturing the 4000 a few months ago because they don't have endless factories. Those resources were reallocated to 5000 series and thus pushed the price for the 4000 up to the ridiculous place it is now (about $2,000 on ebay)
I think the current appetite for crypto and ai is big enough to consume all 4000 and 5000 series cards to a point of scarcity (even 3090s are still fetching about $1000) but there should be a window where things aren't crazy expensive coming up.
There's no evidence supply will continually outstrip demand unless something unusual happens.
All you need is 4x 4090 GPUs to Train Your Own Model -- and $12000 to buy them
Earlier quoted context omitted.
Also (at least in Southern California) electricity prices and how long the rig is on. Not as bad as the initial build cost, but run costs will add up over time.
That is real concern especially 4090 is not power efficient , as a100 and h100, h200. I live in Reno so it was ok
You can also train your own model even without GPUs. Just depends on parameter size.
All you need is 4x 4090 GPUs to Train Your Own Model -- and $12000 to buy them
The last time I checked, a modern Threadripper build is a bit over $10,000. So if you have the budget for that but need something GPU-oriented instead, then I could see that being a reasonable option.
But pricing is okay-ish, have a look at Geohot's Tinybox for turnkey solutions.