Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 49 posts

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#31

Post by hedgehog0 »

janalsncm wrote: post #30
hedgehog0 wrote: post #9

Those who do not recommend, do you have any suggestions for people just getting into the field?

If you’re just getting into the field, by far the easiest thing will be to start out with free Google Colab notebooks until you hit their free tier limits. Kaggle also has free GPUs to play with. When you’re starting out your bottleneck will likely not be your hardware but the learning curve of the content. After that, use a managed service with GPUs. They will keep all of the hardware working, you just pay a monthly…

Thank you. I also saw Colab and Paperspace recommended elsewhere.

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#32

Post by hedgehog0 »

roxgib wrote: post #27
hedgehog0 wrote: post #9

Those who do not recommend, do you have any suggestions for people just getting into the field?

For deep learning/NNs, Google Colab/Kaggle notebooks are where most people start. There are lots of cloud providers that are harder to set up on but might be cheaper/faster for a particular project, but while Colab is limiting in some ways it's dead simple to use so you can focus on learning data science. There's a lot of talk about large models atm, but when you're learning you should probably stick to smaller model…

Yeah currently it’s mostly my hobby and interests, not directly for my work…

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#33

Post by ImprobableTruth »

jphoward wrote: post #7

Multi-GPU training is a double-edged sword. If you are at the stage where you are running your code in a iPython notebook then you are almost certainly not going to benefit from the multiple GPUs, and I strongly suspect you'd be better with fewer and larger GPUs, even if training time is prolonged. The reason I say that is, if we go with PyTorch, you basically have 2 options for multi-GPU training. - DataParallel - w…

DistributedDataParallel (potentially) does both model and data parallelism. Data parallelism is also absolutely used when training large models, it has its downsides, but I don't think there's any way around it if you're training with a large amount of gpus.

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#34

Post by phyalow »

dan1234 wrote: post #11

If you’re just learning, I’d actually look at using a cloud based provider where you can just spin up a gpu vps for the duration of each experiment.

I 100% agree, Google Colab Pro (or whatever they call it now) is a decent first step, for minimal money.

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#35

Post by ckdarby »

aosmith wrote: post #23
JonathanFly wrote: post #20

While it's technically not a bad price in terms of raw compute and memory, the noise from the high speed server fans alone is reason enough not to get this. Never mind the power usage. I was going to say maybe it'd be worth it if they were 24GB GPUs, but I'm not sure you can even use recent Pytorch with cards that old. You'd have to work around the limitations. You don't even need a GPU to learn anyway, you can use t…

Had a poweredge in my apartment in college, can confirm they sound like a leaf blower.

Ha, isn't that what basements are for these days?

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#36

Post by ckdarby »

pavlov wrote: post #8

That K40 setup requires a 1 kW power supply. With energy prices up, that can get pretty expensive in the long run. I suspect a newer Nvidia chip manufactured on a more efficient semiconductor process will deliver the same performance for a fraction of the power consumption. This is often a problem with old hardware. The power efficiency gains from improvements between chip process nodes are so fundamental that it’s h…

Depends on where you live. In Quebec, Canada I'm paying $0.064 to $0.098/kWh

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#38

Post by Rastonbury »

ashirviskas wrote: post #37
langsoul-com wrote: post #25

How much online AI compute can you get for $400. Should be enough for most smaller projects yeah?

Around a month and a half of a VM with T4 on GCP. Depends on what you need.

Lambda labs has A100 for like $1 per hour

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#39

Post by JosephRedfern »

sdenton4 wrote: post #24

For just 5x more cash, you can get two new 3090's... 20k cuda cores, and much more modern, so should be much faster than the core count suggests. Worth considering just building two machines, as well; the ability to train multiple models in parallel is in many cases more valuable than the ability to train one big one.

> For just 5x more cash

I don’t feel the words “just” and “5x more cash” really fit in the same sentence!

Not questioning 3090s being more performant, but that’s a whole lot more money before even thinking about buying the supporting components.

Re: Ask HN: Worth it to buy 4x Nvidia Tesla K40 for AI?

#40

Post by rythie »

The pytorch binaries from pip and conda won’t work on these GPUs, though there are some alternative binaries being maintained that still work: https://blog.nelsonliu.me/2020/10/13/newer-pytorch-binaries-...

The latest Nvidia driver no longer supports the K40, so you’ll have to use version 470 (or lower, officially Nvidia says 460, but 470 seems to work). That supports CUDA 11.4 natively. Newer versions of CUDA 11.x are supported: https://docs.nvidia.com/deploy/cuda-compatibility/index.html though CUDA 12 is not.

In my testing, a system with a single RTX3060 was faster in tensorflow than with 3 K40s and probably close to the performance of 4 k40s.

If you are considering other GPUs, there are some good benchmarks here (The RTX3060 is not there, though the GTX1080Ti was almost the same performance in the tensorflow test they run): https://lambdalabs.com/gpu-benchmarks

As others have said Google CoLab is free option you can use.

Post reply on HN