Earlier quoted context omitted.
Had a poweredge in my apartment in college, can confirm they sound like a leaf blower.
Ha, isn't that what basements are for these days?
That is where my rack is!
Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
41–49 of 49 posts
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#42For some workloads, it’s almost all about the VRAM. In those cases I’ve been wondering if getting a high memory M1 or M2 Mac could be a good lab machine thanks to unified memory. It’ll run more quietly, use significantly less power, no worries about overloading your electric circuit. On a 128 GB RAM Mac Studio you could theoretically run or even train models that otherwise would require multiple $6k A6000 GPUs in custom machine builds taking oodles of power at the plug. It’d be slow but slow beats not possible. And if you need a new development machine anyhow, you can justify some of that beefy Mac Studio’s cost as part of your required spend anyhow. PyTorch has supported “mps” as a target device for some time now.
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#43Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#44Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#45Multi-GPU training is a double-edged sword. If you are at the stage where you are running your code in a iPython notebook then you are almost certainly not going to benefit from the multiple GPUs, and I strongly suspect you'd be better with fewer and larger GPUs, even if training time is prolonged. The reason I say that is, if we go with PyTorch, you basically have 2 options for multi-GPU training. - DataParallel - w…
DistributedDataParallel (potentially) does both model and data parallelism. Data parallelism is also absolutely used when training large models, it has its downsides, but I don't think there's any way around it if you're training with a large amount of gpus.
How does DDP do model parallel?
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#46Earlier quoted context omitted.
DistributedDataParallel (potentially) does both model and data parallelism. Data parallelism is also absolutely used when training large models, it has its downsides, but I don't think there's any way around it if you're training with a large amount of gpus.
How does DDP do model parallel?
I phrased that wrongly, DDP itself doesn't of course. I meant that using it in the way GP does is also doing model parallelism.
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#47Earlier quoted context omitted.
How does DDP do model parallel?
I phrased that wrongly, DDP itself doesn't of course. I meant that using it in the way GP does is also doing model parallelism.
I don’t see any mention of model parallel in GP post. How could you possibly use DDP to enable it?
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#48I bought that system! Hopefully didn't snag it from you. Admittedly don't have a ton of hair in the game yet, but excited to try to get parallel gpus working. Hoping to be able to use it as a render node too and maybe retrain some diffusion models with images of myself. Please link helpful resources if you're able to ;) hehe. You can call me dumb too. I will be mastering in geography starting in the fall, so i'm hoping it will be nice for use with qgis and other gis related projects, rasters are fairly similar to images after all. Again, naive but hey only 500$ in the game after all, would rather take that step than buying a 1k$+ card I might just end up using for minecraft. The fellow said he had to compile cuda from source which is something I am not looking forward to, though I'd like to look into virtualization options for distributed computing... might be something there. Wish me luck! Time to assemble the home lab.
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#49Earlier quoted context omitted.
I phrased that wrongly, DDP itself doesn't of course. I meant that using it in the way GP does is also doing model parallelism.
I don’t see any mention of model parallel in GP post. How could you possibly use DDP to enable it?
GP contrasts DP and DDP by saying that DP is "where you clone your model over each GPU" and DDP is "'proper' multi-GPU training - you can now train big models and put a little bit of data on each GPU". That's simply not what DP or DDP is. What could this possibly mean if it's not misunderstanding DP as data parallelism and DDP as model parallelism? I'm fairly certain that what they're describing is using DDP (which only does data parallelism) in addition to model parallelism.