Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building your own deep learning computer is 10x cheaper than AWS

medium.com

Post reply on HN
Print view View original

111–120 of 269 posts

Re: Building your own deep learning computer is 10x cheaper than AWS

#111

Post by thisisjeffchen »

gnur wrote: post #57

> Even when you shut your machine down, you still have to pay storage for the machine at $0.10 per GB per month, so I got charged a couple hundred dollars / month just to keep my data around. Curious how it relates to sticking only a single terabyte SSD in the machine. As a couple hundred dollars per month should relate to a couple terabytes.

Thanks for the catch! Actually that wasn't intended as a direct comparison to this machine's 1TB storage (I think I had ~2TB in the cloud). I can see how this is confusing - just edited the article.

Re: Building your own deep learning computer is 10x cheaper than AWS

#112

Post by spiritcat »

drewg123 wrote: post #85
VHRanger wrote: post #70

Earlier quoted context omitted.

They seriously can't buy a graphics card and slap it in the PCIe slot?

When I worked at Google, I really missed the dual monitor setup I had had at my previous job. I asked my manager how to get a 2nd monitor. Apparently, since I had the larger monitor, I was not allowed to get a 2nd one without all kinds of hassle. I asked if I was allowed to just buy one from Amazon and plug it in, and I was told no. I finally just grabbed an older one that had been sitting in the hallway of the cube-…

Wow, no dual/triple monitor setup is just uncivilized.

Re: Building your own deep learning computer is 10x cheaper than AWS

#113

Post by leon_sbt »

I ran into this exact issue about 2 years ago about build vs rent. Ultimately I chose build.

Here's my thoughts/background:

Background: Doing small scale training/fine tuning on datasets. Small time commercial applications. I find renting top shelf VM/GPU combos on the cloud to be psychologically draining. Did I forget to shut off my $5 dollar an hour VM during my weekend camping trip? I hate it when I ask myself questions like that.

I would rather spend the $2k upfront and ride the depreciation curve, than have the "constant" VM stress. Keep in mind, this is for a single instance, personal/commercial use rig.

I feel that DL compute decisions aren't black/white and should be approached in stages.

Stage 0: If you do full time computer work at a constant location, you should try to own a fast computing rig. DL or not. Having a brutally quick computer makes doing work much less fatiguing.Plus it opens up the window to experimenting with CAD/CAE/VFX/Photogrammetry/video editing. (4.5ghz i7 8700k +32gb ram +SSD)

Stage 1: Get a single 11/12 gb GPU. 1080TI or TitanX (Some models straight up won't fit on smaller cards). Now you can go on Github and play with random models and not feel guilty about spending money on a VM for it.

Stage 2: Get a 2nd GPU. Makes for writing/debugging multi-gpu code much easier/smoother.

Stage 3: If you need more than 2 GPU's for compute, write/debug the code locally on your 2 GPU rig. Then beam it up to the cloud for 2+ gpu training. Use preemptible instances if possible for cost reasons.

Stage 4: You notice your cloud bill is getting pretty high($1k+ month) and you never need more than 8x GPUs for anything that your doing. Start the build for your DL runbox #2. SSH/Container workloads only. No GUI, no local dev. Basically server grade hardware with 8x GPUS.

Stage 5: I'm not sure, don't listen to me :)

Re: Building your own deep learning computer is 10x cheaper than AWS

#114

Post by supernovae »

ethbro wrote: post #79
mlthoughts2018 wrote: post #45

Earlier quoted context omitted.

In most companies, AWS just becomes a new front-end to the same old IT bureaucracy, and dev teams are still disallowed from creating their own instances or EMR clusters or setting up new products like their own Redshift instance or ECR deployment solution.

Respectfully, those companies' cloud architects suck. If someone goes to the trouble to migrate onto cloud, and then replicates pre-devops workflows... wow.

Not every company is a single page webapp and simple service portfolio... I work at a place with 3,000+ developers and over 700 applications - there is no way in hell our cloud portfolio would have any standards if we didn't have a robust operations/engineering team making it work. Sometimes, even when you adopt the cloud, you realize that your operations model is even more important and there is nothing wrong with that.

The real problem is believing that there is a pure model that works for everyone.

Re: Building your own deep learning computer is 10x cheaper than AWS

#115

Post by thisisjeffchen »

paradite wrote: post #78

This works if your machine learning is just for data analytics purposes. The article somehow completely ignores integration with other services to query or update the model, which requires some API to be hosted on a static ip or domain, as well as the devops process.

Yeah the main use case here is training, which is much more of an offline process. For inference, you will probably want a cloud provider. Though the cost difference still holds, so maybe a startup idea.

Re: Building your own deep learning computer is 10x cheaper than AWS

#116

Post by xfitm3 »

LeonM wrote: post #3

Own hardware is always cheaper to buy than using a cloud service, but keeping it running 24/7 involves substantial costs. Sure, if you run a solo operation, you can just get up during the night to nurse your server, but at some point that no longer makes sense to do. Somewhere along the way we forgot about this and it's now perfectly normal to run a blog on a GKE 3 VM kubernetes cluster, costing 140 EUR/month.

I think the real answer to this question is unhelpful, which is: it depends.

I ran a detailed cost analysis of tier 3 onprem vs aws about 7 years ago. I included the cost of maintaining servers, support staff salaries, rent, insurance, employee dwell time etc and onprem was still cheaper. Maybe it's different now.

We put significant thought into being cheap. I think constraint can breed innovation.

Re: Building your own deep learning computer is 10x cheaper than AWS

#117

Post by davidkuhta »

thisisjeffchen wrote: post #104
acd10j wrote: post #92

Earlier quoted context omitted.

Most of time standard power supplies and cabinet of these Desktop does not support 1080Ti power consumption.

Hey there, I'm the writer for the article. I have another article planned that will talk about part picking and actual building. For now though, here is the parts list: https://pcpartpicker.com/b/B6LJ7P w/ 1600W psu.

Great read! As an aside your hyperlink to HomebrewAIClub accidentally included a comma.

Re: Building your own deep learning computer is 10x cheaper than AWS

#118

Post by tyingq »

ovi256 wrote: post #87
anaxag0ras wrote: post #72

You can also get GPU instances at much cheaper rates from Hetzner and OVH. https://www.hetzner.de/dedicated-rootserver/ex51-ssd-gpu https://www.ovh.com/world/public-cloud/instances/prices/

Only in non-North American datacenters. In NA, Nvidia can enforce their driver license, which prohibits use of consumer GPUs in datacenters. A nice advantage of non-consumer GPUs is their bigger RAM size. Consumer GPUs, even the newest 2080 Ti, has only 11 GB. Datacenter GPUs have 16GB or 32 GB (V100). This is important for very big models. Even if the model itself fits, small memory size forces you to reduce batch s…

"Only in non-North American datacenters"

OVH offers these in their Canadian data center.

Re: Building your own deep learning computer is 10x cheaper than AWS

#119

Post by mastax »

tfolbrecht wrote: post #46
psergeant wrote: post #4

> Nvidia contractually prohibits the use of GeForce and Titan cards in datacenters. So Amazon and other providers have to use the $8,500 datacenter version of the GPUs, and they have to charge a lot for renting it.

Curious of the MTBF (mean time between failure) rate of a GefForce/Titan series GPU under continuous utilization in datacenter conditions vs a desktop computer with intermittent usage. I don't want to believe Nvidia is just out to stiff cloud providers. Maybe it's to protect themselves from warranty abuse?

It's market segmentation. It doesn't need to be more than that

Re: Building your own deep learning computer is 10x cheaper than AWS

#120

Post by maaark »

> There’s one 1080 Ti GPU to start (you can just as easily use the new 2080 Ti for Machine Learning at $500 more — just be careful to get one with a blower fan design)

I don't believe there are any blower-style 20-series cards. The reference cards use a dual-fan design.

Post reply on HN