Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building your own deep learning computer is 10x cheaper than AWS

medium.com

Post reply on HN
Print view View original

171–180 of 269 posts

Re: Building your own deep learning computer is 10x cheaper than AWS

#171

Post by chaosbutters »

leon_sbt wrote: post #113

I ran into this exact issue about 2 years ago about build vs rent. Ultimately I chose build. Here's my thoughts/background: Background: Doing small scale training/fine tuning on datasets. Small time commercial applications. I find renting top shelf VM/GPU combos on the cloud to be psychologically draining. Did I forget to shut off my $5 dollar an hour VM during my weekend camping trip? I hate it when I ask myself que…

upgrade to the TR 2990wx and home CAD/CAE/VFX/Photogrammetry/video editing becomes super nice

Re: Building your own deep learning computer is 10x cheaper than AWS

#172

Post by mlthoughts2018 »

tetha wrote: post #164
askmike wrote: post #86

Earlier quoted context omitted.

This has less to do with cloud vs own hardware and more about how the company is structured. I've worked in companies before with an ops department: a few people responsible for managing all the cloud servers. All the devs (like me at the time) work locally and have access to an isolated dev/uat environment provided by that team. That team had most of the show automated, they weren't really provisioning any machines…

I dunno, that sounds like the a good way to do it. I kinda shake my head if people call for devs to handle all the infrastructure and everything in prod. Why should a developer concern himself with details of scaling and tuning postgres, elasticsearch or loadbalancers? If you don't outsource that, that's the job for the ops team. However, if that's the responsibility of the ops team, there's no reason for dev to have…

I’m a machine learning engineer. I’d love it if I only needed to worry about machine learning and application interfaces.

Instead, because ops / infra arbitrarily block me from what I need, I have to be an expert on database internals, network bottlenecks, app security topics, deployment, containers, CI tools, etc. etc., both so that I can “do it myself” when ops e.g. refuses to acknowledge some assumption-breaking GPU architecture we need, and so that I can exhaustively deal with every single arch / ops debate or bureaucratic hurdle that comes up for me to endlessly justify everything I need to do far beyond any reasonable standard.

For me, managing devops shit myself is a necessary evil. Far better than the case of unresponsive / bureaucratic ops teams, but worse than the unicorn case of an actual customer service oriented ops team that actually cares rather than engages in convenience-for-themselves optimization at every turn.

Re: Building your own deep learning computer is 10x cheaper than AWS

#174

Post by spullara »

unbearded wrote: post #165
dagw wrote: post #29

You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)

Unfortunately, the fight goes even further than that when you go against the cloud. Last week I was in an event with the CTOs of many of the hottest startups in America. It was shocking how much money is wasted on the cloud because inefficiencies and they simply don't care how much it costs. I guess since they are not wasting their own money, they can always come up with the same excuse: developers are more expensive…

Only a Rails developer would think 6k requests per minute with 40ms latency is reasonable with all that hardware. If you rewrote it you probably only need 1 server but you will probably make an argument about how developer time is more valuable :)

Re: Building your own deep learning computer is 10x cheaper than AWS

#175

Post by phmagic »

dkobran wrote: post #127

NVIDIA is attempting to separate enterprise/datacenter and consumer chips to justify the cost disparity. Specifically, they're introducing memory, precision etc. limits which have major performance implications to GeForce and there's also the EULA which was been mentioned here. That said, everything AWS comes at a premium as they're making the case that on-demand scale outweighs the pain of management/CapEx. This pre…

I contemplated building my own machine for deep learning. I'm a geek so I do like building hardware. But I couldn't even justify spending $2k on a deep learning machine.

Paperspace has eliminated my desire to build a deep learning computer thanks to their insanely low prices.

Current prices are around $0.78 per hour for a Nvidia Quadro P5000, that's pretty comparable to a 1080 TI.

On top of that you can even run Gradient notebooks (on demand) without even setting up a server. This is the future when bandwidth costs are minimal: thin clients, powerful servers.

At the end of the day, I wanted to spend more time tuning the ML pipeline rather than fussing with drivers, OS dependencies, etc

Sure there's lots of things that Paperspace could do better, but their existing product is already leaps and bounds better than GCloud or AWS. AWS and GCloud wins through big contracts with large businesses and I'm just a little guy.

Disclosure: I do not work for Paperspace and am not paid to endorse them in any way. I love their product.

Re: Building your own deep learning computer is 10x cheaper than AWS

#177

Post by lotyrin »

scurvy wrote: post #151
nabla9 wrote: post #122

Earlier quoted context omitted.

That's very passive aggressive way to deal with it. If that's your only option you are really a cubicle slave in corporate hell. In my opinion it's better to escalate upwards with proposals and not back down easily. You just have to frame it correctly and use right names and terms. * Usually big companies understand the concept of "a lab" that has infrastructure managed outside the corporate IT. Once you fight the ha…

In my experience, the blind drive to the cloud comes from above. It's someone's pet project and you're either on-board or not. I won't go as far as calling it yet another cyclical IT phase, but it has all the hallmarks of one.

It has to be, even with as irrational as the market can be I don't see people sustaining this kind of spend on public clouds -- the cultural barriers and perverse incentives that prevent effective use of private / hybrid cloud have to erode eventually.

Re: Building your own deep learning computer is 10x cheaper than AWS

#178

Post by captain_crabs »

spullara wrote: post #174
unbearded wrote: post #165

Earlier quoted context omitted.

Unfortunately, the fight goes even further than that when you go against the cloud. Last week I was in an event with the CTOs of many of the hottest startups in America. It was shocking how much money is wasted on the cloud because inefficiencies and they simply don't care how much it costs. I guess since they are not wasting their own money, they can always come up with the same excuse: developers are more expensive…

Only a Rails developer would think 6k requests per minute with 40ms latency is reasonable with all that hardware. If you rewrote it you probably only need 1 server but you will probably make an argument about how developer time is more valuable :)

It's a matter of dosage. You might be talking about how another tech stack would perform better at this metric, but the price of that is the company would have been simply unable to ship anything in the time they could afford.

Swinging the conversation beyond the dosages of either side doesn't produce interesting insight

Re: Building your own deep learning computer is 10x cheaper than AWS

#179

Post by oneshot908 »

tfolbrecht wrote: post #46
psergeant wrote: post #4

> Nvidia contractually prohibits the use of GeForce and Titan cards in datacenters. So Amazon and other providers have to use the $8,500 datacenter version of the GPUs, and they have to charge a lot for renting it.

Curious of the MTBF (mean time between failure) rate of a GefForce/Titan series GPU under continuous utilization in datacenter conditions vs a desktop computer with intermittent usage. I don't want to believe Nvidia is just out to stiff cloud providers. Maybe it's to protect themselves from warranty abuse?

The failure rate of GeForce is higher than Tesla (though it's not that much worse in my experience these days). OEMs worked around this by burning the GPUs in for a couple days running HPC or DL code before sending them out to customers and then they binned the failed GPUs into gaming machines where bitwise reproducibility didn't matter as much. Also, it's usually the memory controller that's defective on 1080TIs so it's pretty easy to spot early.

NVIDIA was not amused.

Re: Building your own deep learning computer is 10x cheaper than AWS

#180

Post by Walkman »

Every hardware can be cheaper than AWS if you build it yourself, that's not the point. The ongoing cost of maintaining and improving it, and the necessary knowledge might not. Also instead of building your own hardware, you can build the app instead. I thought these are obvious for anyone.
Post reply on HN