If you want to learn about which GPUs make sense to buy or rent for various ML workload, i highly recommend this Tim Dettmers article:
https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...
Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
21–30 of 49 posts
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#22Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#23While it's technically not a bad price in terms of raw compute and memory, the noise from the high speed server fans alone is reason enough not to get this. Never mind the power usage. I was going to say maybe it'd be worth it if they were 24GB GPUs, but I'm not sure you can even use recent Pytorch with cards that old. You'd have to work around the limitations. You don't even need a GPU to learn anyway, you can use t…
Had a poweredge in my apartment in college, can confirm they sound like a leaf blower.
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#24For just 5x more cash, you can get two new 3090's... 20k cuda cores, and much more modern, so should be much faster than the core count suggests.
Worth considering just building two machines, as well; the ability to train multiple models in parallel is in many cases more valuable than the ability to train one big one.
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#25How much online AI compute can you get for $400.
Should be enough for most smaller projects yeah?
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#26Those who do not recommend, do you have any suggestions for people just getting into the field?
For Stable Diffusion inference I recommend an nvidia 3060. If you want to train your own stable diffusion models I recommend a 3090. If you want to work with LLMs I recommend 2x3090.
I also recommend at least 64 GB of RAM, and at least 2 TB of storage. It seems like overkill at first but man do the GBs disappear like magic.
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#27Those who do not recommend, do you have any suggestions for people just getting into the field?
For deep learning/NNs, Google Colab/Kaggle notebooks are where most people start. There are lots of cloud providers that are harder to set up on but might be cheaper/faster for a particular project, but while Colab is limiting in some ways it's dead simple to use so you can focus on learning data science.
There's a lot of talk about large models atm, but when you're learning you should probably stick to smaller models because the training time drastically reduces how fast you can iterate, and most jobs aren't NLP based anyway.
Remember also that 99% of problems classical models are faster and just as good or even better than neural nets. Of course if you're interested in neural nets as a hobby, go for it, they're truly fascinating and you can do some amazing things just in Colab, but they still aren't the best option for most real world problems.
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#28Well thank you all for the suggestions, I think I mostly agree it’s not a good idea. From power cost/consumption, to lack of modern CUDA, PyTorch support as well as the complexity with parallel compute really put a dent in the perceived value.
I may have to just save up and get a more capable card with more VRAM. I still want to learn how to do parallel compute but I realize I could just do that at any other time and doesn’t have to be hardware that I necessarily own (rent out a cloud server) even though that would be really nice.
Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#29Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?
#30Those who do not recommend, do you have any suggestions for people just getting into the field?
If you’re just getting into the field, by far the easiest thing will be to start out with free Google Colab notebooks until you hit their free tier limits. Kaggle also has free GPUs to play with. When you’re starting out your bottleneck will likely not be your hardware but the learning curve of the content.
After that, use a managed service with GPUs. They will keep all of the hardware working, you just pay a monthly fee. I have no interest in diverting mental energy from learning ML to how to build a computer so that’s what I do. I use Paperspace but it has some very annoying issues so I can’t fully endorse it.