111–120 of 279 posts
I wonder how the cost compares to a Tinybox. $25k for 6x 4090 or $15k for 6x 7900XTX. Of course that's the full package with power supplies, CPU, storage, cooling, assembly, shipping, etc. And a tested, known good hardware/software configuration which is crucial with this kind of thing.
I wonder how the cost compares to a Tinybox. $25k for 6x 4090 or $15k for 6x 7900XTX. Of course that's the full package with power supplies, CPU, storage, cooling, assembly, shipping, etc. And a tested, known good hardware/software configuration which is crucial with this kind of thing.
You could just buy a Mac Studio for 6500 USD, have 192 GB of unified RAM and have way less power consumption.
I know it's a fraction of the size, but my 32GB studio gets wrecked by these types of tasks. My experience is that they're awesome computers in general, but not as good for AI as people expect. Running llama3.1 70B is brutal on this thing. Responses take minutes. Someone running the same model on 32GB of GPU memory seems to have far better results from what I've read.
Earlier quoted context omitted.
The main thing stopping me from going beyond 2x 4090’s in my home lab is power. Anything around ~2k watts on a single circuit breaker is likely to flip it, and that’s before you get to the costs involved of drawing that much power for multiple days of a training run. How did you navigate that in a (presumably) residential setting?
>Anything around ~2k watts on a single circuit breaker is likely to flip it I'm curious, how do you use e.g. a washing machine or an electric kettle, if 2kW is enough to flip your breaker? You should simply know your wiring limits. Breaker/wiring at my home won't even notice this.
Earlier quoted context omitted.
>Anything around ~2k watts on a single circuit breaker is likely to flip it I'm curious, how do you use e.g. a washing machine or an electric kettle, if 2kW is enough to flip your breaker? You should simply know your wiring limits. Breaker/wiring at my home won't even notice this.
Americans do not have electric kettles and need special circuits for electric clothes dryers.
You're correct that the dryer is on a larger circuit, though.
Earlier quoted context omitted.
The main thing stopping me from going beyond 2x 4090’s in my home lab is power. Anything around ~2k watts on a single circuit breaker is likely to flip it, and that’s before you get to the costs involved of drawing that much power for multiple days of a training run. How did you navigate that in a (presumably) residential setting?
>Anything around ~2k watts on a single circuit breaker is likely to flip it I'm curious, how do you use e.g. a washing machine or an electric kettle, if 2kW is enough to flip your breaker? You should simply know your wiring limits. Breaker/wiring at my home won't even notice this.
Boiling 1 liter takes like 2 mins. Most Americans don’t have kettles because they don’t drink tea.
Earlier quoted context omitted.
>Anything around ~2k watts on a single circuit breaker is likely to flip it I'm curious, how do you use e.g. a washing machine or an electric kettle, if 2kW is enough to flip your breaker? You should simply know your wiring limits. Breaker/wiring at my home won't even notice this.
Americans do not have electric kettles and need special circuits for electric clothes dryers.
Earlier quoted context omitted.
The main thing stopping me from going beyond 2x 4090’s in my home lab is power. Anything around ~2k watts on a single circuit breaker is likely to flip it, and that’s before you get to the costs involved of drawing that much power for multiple days of a training run. How did you navigate that in a (presumably) residential setting?
I can't believe a group of engineers are so afraid of residential power. It is not expensive, nor is it highly technical. It's not like we're factoring in latency and crosstalk... Read a quick howto, cruise into Home Depot and grab some legos off the shelf. Far easier to figure out than executing "hello world" without domain expertise.
Earlier quoted context omitted.
The main thing stopping me from going beyond 2x 4090’s in my home lab is power. Anything around ~2k watts on a single circuit breaker is likely to flip it, and that’s before you get to the costs involved of drawing that much power for multiple days of a training run. How did you navigate that in a (presumably) residential setting?
This is funny as a european, since we have many, many groups where we reguarly will run 2kW, and some, loads. Really no issue, but I guess lower voltage makes it a problem.