Building your own deep learning computer is 10x cheaper than AWS
1–10 of 269 posts
Re: Building your own deep learning computer is 10x cheaper than AWS
#2I really want to believe this. Of course the numbers given depend on very frequent use of your machine, but still. One would imagine that building a datacenter at scale and only renting when you actually are training a model to be much cheaper, but the reality appears to be not so.
So where does the money go?
Three places:
- AWS/Google/Whoever-you're-renting-from obviously get a cut
- Inefficiencies in the process (there's lots of engineers and DB administrators and technicians and and and people who have to get paid in the middle.)
- Thirdly, and this is what most surprised me, NVIDIA takes a big cut. Apparently the 1080Ti and similar cards are consumer only, whilst datacenters & cloud providers have to buy their Tesla line of cards, with corresponding B-to-B support and price tag (3k-8k per card). [1]
So, given these three money-gobbling middlemen, it does seem to kinda make sense to shell out 3.000$ for your own machine, if you are serious about ML.
Some small additional upsides are that you get a blazing fast personal PC and can probably run Crysis 3 on it.
[1]https://www.cnbc.com/2017/12/27/nvidia-limits-data-center-us...
Re: Building your own deep learning computer is 10x cheaper than AWS
#3Own hardware is always cheaper to buy than using a cloud service, but keeping it running 24/7 involves substantial costs.
Sure, if you run a solo operation, you can just get up during the night to nurse your server, but at some point that no longer makes sense to do.
Somewhere along the way we forgot about this and it's now perfectly normal to run a blog on a GKE 3 VM kubernetes cluster, costing 140 EUR/month.
Re: Building your own deep learning computer is 10x cheaper than AWS
#4> Nvidia contractually prohibits the use of GeForce and Titan cards in datacenters. So Amazon and other providers have to use the $8,500 datacenter version of the GPUs, and they have to charge a lot for renting it.
Re: Building your own deep learning computer is 10x cheaper than AWS
#5You know, all the negatives of building your own machine assumes you run it 24/7, and we sure as hell don't. We run these models maybe a couple of times a week, but we expect fast results when we do, and that's unchanged.
I will bring this up with the rest of my data engineering team, this might be a good idea.
Re: Building your own deep learning computer is 10x cheaper than AWS
#6Of course building is cheaper than AWS...
It also means you need to build the computer, maintain it, upgrade it, store it somewhere, connect it to the internet, and so on.
And you can't just scale it if you need to.
I mean, the cloud's here for a reason...
Re: Building your own deep learning computer is 10x cheaper than AWS
#7You know, all the negatives of building your own machine assumes you run it 24/7, and we sure as hell don't. We run these models maybe a couple of times a week, but we expect fast results when we do, and that's unchanged. I will bring this up with the rest of my data engineering team, this might be a good idea.
> Assuming your 1 GPU machine depreciates to $0 in 3 years (very conservative), the chart below shows that if you use it for up to 1 year, it’ll be 10x cheaper, including costs for electricity
You can invert that to 3 years of use at 33% utilization which would come out cheaper (or is my maths broken?). Still doesn't sound like it'd be a good match for your usage though.
That 3 years is extremely conservative though, in reality it would probably be much longer and there are some potential upgrade paths to factor in as well. Not to mention the potential to use it for other purposes.
Re: Building your own deep learning computer is 10x cheaper than AWS
#8GPU accelerates training time considerably. For development it's very convenient to use a Linux laptop with nVidia Optimus. I was able to train a quite complex model on laptops's nVidia (optirun process) while continue working using Intel SOC's GPU. However GPU memory is crucial it's better to choose a laptop with 4GB or more video RAM.
Re: Building your own deep learning computer is 10x cheaper than AWS
#9> Nvidia contractually prohibits the use of GeForce and Titan cards in datacenters. So Amazon and other providers have to use the $8,500 datacenter version of the GPUs, and they have to charge a lot for renting it.
How can Nvidia anymore enforce that than movie producers can stop rentals of physical media.
Re: Building your own deep learning computer is 10x cheaper than AWS
#10GPU accelerates training time considerably. For development it's very convenient to use a Linux laptop with nVidia Optimus. I was able to train a quite complex model on laptops's nVidia (optirun process) while continue working using Intel SOC's GPU. However GPU memory is crucial it's better to choose a laptop with 4GB or more video RAM.
I've found that many models require 8GB or more; the T1080 cards are "just about" good enough