Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linode GPU Instances

blog.linode.com

Post reply on HN
Print view View original

41–50 of 75 posts

Re: Linode GPU Instances

#41

Post by jamesblonde »

Who_me wrote: post #34

Hey peeps full disclosure I work as one of Linode's RnD engineers. I want to try to get to as many of these as I can. One of the biggest questions is why the Quadro RTX 6000? Few things: 1. Cost it has the same performance as the 8000. The difference is 8 more GB of RAM that comes at a steep premium. Cost is important to us as it allows us to be at a more affordable price point. 2. We have all heard or used the Tesla…

If you really want low cost to compute for Deep Learning and you needs lots of compute and don't want to pay for V100s, then the AMD Vega R7 is the card for you. 700 dollars, 16GB Ram, 1TB of GPU bandwidth (higher than the V100!), works with Tensorflow (pip install tensorflow-rocm), and about 60% of the performance on resnet-50.FP64 is not fully gimped (it is halved, i think - so still quite good). Put lots of them in servers with PCI 4.0, and you can do great things. Here's a recent talk on it:

https://www.youtube.com/watch?v=neb1C6JlEXc

Re: Linode GPU Instances

#42

Post by jamesblonde »

coherentpony wrote: post #9

Does anybody know if there are any cloud instances with AMD GPUs?

GPUEater do, i think. Right now, though, they are a viable option for an on-premise use case where you have a budget of say a $100k dollars or more and need a huge amount of compute and have larger models to train. The Vega R7 gives you 16GB Ram (11GB in the 2080Ti) and is just slightly lower performance than the 2080Ti (322 vs 302 images/sec for resnet-50 from here: https://www.youtube.com/watch?v=neb1C6JlEXc ). And you have servers with PCI-4.0 support, so that distributed training scales (yes, 2080-Ti supports nvlink, but nvlink servers cost way more). Simple math example. A PCI-4.0 server with 256GB Ram and 8xVegaR7 should cost around $10K. With a couple of switches and racks, you can get 100s of GPUs for just a couple of hundred thousand dollars (note, only 2 GPU servers per rack for now is normal, otherwise you have to buy non-commodity racks with high power draw).

Re: Linode GPU Instances

#43

Post by microtonal »

jamesblonde wrote: post #41
Who_me wrote: post #34

Hey peeps full disclosure I work as one of Linode's RnD engineers. I want to try to get to as many of these as I can. One of the biggest questions is why the Quadro RTX 6000? Few things: 1. Cost it has the same performance as the 8000. The difference is 8 more GB of RAM that comes at a steep premium. Cost is important to us as it allows us to be at a more affordable price point. 2. We have all heard or used the Tesla…

If you really want low cost to compute for Deep Learning and you needs lots of compute and don't want to pay for V100s, then the AMD Vega R7 is the card for you. 700 dollars, 16GB Ram, 1TB of GPU bandwidth (higher than the V100!), works with Tensorflow (pip install tensorflow-rocm), and about 60% of the performance on resnet-50.FP64 is not fully gimped (it is halved, i think - so still quite good). Put lots of them i…

If you really want low cost to compute for Deep Learning and you needs lots of compute and don't want to pay for V100s, then the AMD Vega R7 is the card for you. 700 dollars, 16GB Ram, 1TB of GPU bandwidth (higher than the V100!), works with Tensorflow (pip install tensorflow-rocm), and about 60% of the performance on resnet-50.FP64 is not fully gimped (it is halved, i think - so still quite good).

Two of my colleagues use high-end AMD GPUs to train RNNs and transformers with tensorflow-rocm. There are still some nasty bugs (e.g. [1]), so it is currently not for everyone. However, given how far they have come compared to 1-2 years ago, it is very likely that in a year or so, they are a real competitor to NVIDIA for compute. That competition was long needed.

[1] https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/...

Re: Linode GPU Instances

#44

Post by latchkey »

walrus01 wrote: post #16
MuffinFlavored wrote: post #10

Can these be used for crypto mining at any level of efficiency? I was able to mine GRLC back in the day on AWS spot instances at a VERY mild degree of profitability.

not really, most cryptocurrency is at the stage where the only thing effective is a combination of custom ASICs and nearly free electricity. About twelve months ago I looked into mining ethereum with state of the art GPUs and it would not have had a reasonable ROI unless I was literally paying $0.00 per kWh. And that was before its value per coin dropped a lot.

When the value dropped, the network hashrate dropped and difficulty went down so things actually became profitable again.

The best time to mine is during the drops, not the highs, unless you follow buy high, sell low and don't believe the market will correct for the better again (which it has).

Of course, it depends on electricity prices, but it is profitable to mine ethereum, especially if you know how to tune the cards to maximize hash/consumption.

That said, mining is competitive and difficult and unless you are going to go really large, don't bother. If you are interested in learning about it, definitely experiment though don't expect to make a lot of money.

Re: Linode GPU Instances

#46

Post by cstejerean »

krick wrote: post #40
tootahe45 wrote: post #32

Earlier quoted context omitted.

I don't have specific experience with ML, but AWS spot pricing is by far the best deal last time i checked for GPU. You can get something much more powerful than a gtx1080 and get your task done more quickly. The downside is that at any time your instance can be shut down after a short warning signal to backup your progress, so it may or may not be suitable for what you're doing.

Does the price actually depend on whether you are using a GPU or simply an instance you choose? Let's say you need to do some work that will require a GPU, so you spend 5 hours setting up an environment, doing some light programming/experiments in an Jupyter notebook, downloading datasets, looking at the data. Then you train for an hour then one more hour looking at the data, drinking coffee, stuff like that. Then tr…

If you use a GPU instance you pay the cost for it whether not you use the actual GPU. If the GPU time is short relative to the other stuff you are doing (like data cleanup) it might make sense to do your non-GPU related setup on a different instance first.

Re: Linode GPU Instances

#47

Post by tarasmatsyk »

picozeta wrote: post #19

I would go with Hetzner: https://www.hetzner.com/dedicated-rootserver/ex51-ssd-gpu GTX1080 for 100$ a month. Grantend, it is older, but it still works for DL. Let's say you do 10 experiments a month for ~20 hours. Thats 0.5$/hour and I don't think it is 3 times faster. If you then want to do even more learning the price goes even down. //DISCLAIMER: I do not work for them, but used it for DL in the past and it was fo…

Agree, had exactly the same experience.

It is not a server card, however, it is much faster than any old AWS instances for 1k$/m (if you happen to be an AWS user and did not want to upgrade because of the price going up 3x) TBH, 100 bucks per month is free, while most of the researches do not have 1k$/m for a server, it is cheaper to buy hardware and put Linux on it.

There are of course other options and Linode is kinda late to the party, but I am happy they made this move.

Re: Linode GPU Instances

#48

Post by tarasmatsyk »

perennate wrote: post #11
thenightcrawler wrote: post #8

Isn't AWS cheaper? edit: could be wrong thought I read of AWS being .65 dollars an hour for deep learning GPU use. edit2: Did a quick look, the .65 dollars doesn't include the actual instance, so its around 1.8 an hour on the low end, I think this cheaper.

p2.xlarge comes with an NVIDIA Tesla K80 GPU for $0.90/hr, but this is now an "old" GPU and the RTX Quadro 6000 should have much higher performance (but I was unable to find any machine learning benchmarks). p3.2xlarge has NVIDIA Tesla V100 GPU which is NVIDIA's most recent deep learning GPU, but it's $3.06/hr. That said, AWS is among the most expensive providers if you just need a deep learning GPU (but obviously AW…

K80 is the crappiest cards I've used last year. For sure it was a good choice 2 years ago, but now you are better to upgrade as any new desktop card is better than K80

Re: Linode GPU Instances

#49

Post by tarasmatsyk »

thenightcrawler wrote: post #8

Isn't AWS cheaper? edit: could be wrong thought I read of AWS being .65 dollars an hour for deep learning GPU use. edit2: Did a quick look, the .65 dollars doesn't include the actual instance, so its around 1.8 an hour on the low end, I think this cheaper.

It depends, for full-time usage, it is a bit more expensive, I think it is a matter of a few hundred, probably less. We've happily migrated from AWS as only one GPU instance cost us near 1k/m. BTW, the newest and the only available GPU instances now should be better RTX6000 even being more expensive.

Re: Linode GPU Instances

#50

Post by jamesblonde »

microtonal wrote: post #43
jamesblonde wrote: post #41

Earlier quoted context omitted.

If you really want low cost to compute for Deep Learning and you needs lots of compute and don't want to pay for V100s, then the AMD Vega R7 is the card for you. 700 dollars, 16GB Ram, 1TB of GPU bandwidth (higher than the V100!), works with Tensorflow (pip install tensorflow-rocm), and about 60% of the performance on resnet-50.FP64 is not fully gimped (it is halved, i think - so still quite good). Put lots of them i…

If you really want low cost to compute for Deep Learning and you needs lots of compute and don't want to pay for V100s, then the AMD Vega R7 is the card for you. 700 dollars, 16GB Ram, 1TB of GPU bandwidth (higher than the V100!), works with Tensorflow (pip install tensorflow-rocm), and about 60% of the performance on resnet-50.FP64 is not fully gimped (it is halved, i think - so still quite good). Two of my colleagu…

Agreed, it is not quite prime-time yet. They are trying to upstream all the ROCm stuff in TensorFlow, and when it gets into mainline and stabilizes, i agree that it has great potential for take-off - particularly from price-sensitive researchers and large companies who need huge GPU farms.
Post reply on HN