Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building your own deep learning computer is 10x cheaper than AWS

medium.com

Post reply on HN
Print view View original

231–240 of 269 posts

Re: Building your own deep learning computer is 10x cheaper than AWS

#231

Post by vl »

xienze wrote: post #192
vl wrote: post #190

Earlier quoted context omitted.

I have a hilarious story about this from Google: I wanted second 30" monitor, so I filed a ticket. They sent me long email listing reasons why I shouldn't get a second monitor, including (numbers are approximate, employee count from 2013 or so) "If every googler gets an extra monitor, in a year it would be equivalent to driving Toyota Camry for 18000 miles." I'm thinking "this can't possible be right", so I spend som…

> They sent me long email listing reasons why I shouldn't get a second monitor I had no idea Google was so cheap. Gourmet breakfast, lunch, and dinner every day? No problem. A couple hundred bucks for a second monitor? Uh... it's not about the money, we're, uh, concerned about the environment.

To be fair back then decent 30" monitors were around $800-$1000. (Also, it's just normal food, not "gourmet".)

Re: Building your own deep learning computer is 10x cheaper than AWS

#232

Post by cameldrv »

oneshot908 wrote: post #137

It's been this way since day 1. NVLINK remains the only real Tesla differentiator (although mini NVLINK is available on the new Turing consumer GPUs so WTFever). But because none of the DL frameworks support intra-layer model parallelism, all of the networks we see tend to run efficiently in data parallel because doing anything else makes them communication-limited, which they aren't because data scientists end up bu…

I think that the reason no one implements Krizhevsky's OWT (at least in normal training scripts, there's nothing stopping you from doing this in TensorFlow) is that the model parallelism in OWT is only useful where you have more weights than inputs/outputs to a layer. This was true for the FC layers in AlexNet, but hardly anyone uses large FC layers anymore.

Re: Building your own deep learning computer is 10x cheaper than AWS

#233

Post by kwillets »

leon_sbt wrote: post #113

I ran into this exact issue about 2 years ago about build vs rent. Ultimately I chose build. Here's my thoughts/background: Background: Doing small scale training/fine tuning on datasets. Small time commercial applications. I find renting top shelf VM/GPU combos on the cloud to be psychologically draining. Did I forget to shut off my $5 dollar an hour VM during my weekend camping trip? I hate it when I ask myself que…

I'm thinking of starting a prepaid cloud service -- once your $20 is gone it shuts everything off.

Re: Building your own deep learning computer is 10x cheaper than AWS

#234

Post by bogomipz »

uryga wrote: post #209
bogomipz wrote: post #106

>"Nvidia contractually prohibits the use of GeForce and Titan cards in datacenters. So Amazon and other providers have to use the $8,500 datacenter version of the GPUs, and they have to charge a lot for renting it." I wonder if someone might provide some clarification on this. Is this to say only if a reseller buys directly from Nvidia they are compelled by some agreement they signed with Nvidia? How else would this…

Another comment in this thread said that it's due to the license on Nvidia's drivers. So technically you can use the hardware in a datacenter, just not with the official drivers. Unfortunately it seems that the open-source drivers aren't usable for most datacenter purposes, so this effectively limits how you can use the hardware (at least in North America, where they can enforce it).

Wow so the drivers serve as a form of license key. I wonder how long before AWS develops its own GPUs.

Re: Building your own deep learning computer is 10x cheaper than AWS

#235

Post by mr_toad »

gnur wrote: post #57

> Even when you shut your machine down, you still have to pay storage for the machine at $0.10 per GB per month, so I got charged a couple hundred dollars / month just to keep my data around. Curious how it relates to sticking only a single terabyte SSD in the machine. As a couple hundred dollars per month should relate to a couple terabytes.

And assuming you keep it in block storage. If you keep the data in S3 and copy it to block storage as required it’s a lot cheaper.

Re: Building your own deep learning computer is 10x cheaper than AWS

#236

Post by mr_toad »

thisisjeffchen wrote: post #115
paradite wrote: post #78

This works if your machine learning is just for data analytics purposes. The article somehow completely ignores integration with other services to query or update the model, which requires some API to be hosted on a static ip or domain, as well as the devops process.

Yeah the main use case here is training, which is much more of an offline process. For inference, you will probably want a cloud provider. Though the cost difference still holds, so maybe a startup idea.

Inference can usually be done on much cheaper hardware, or using lambda.

Re: Building your own deep learning computer is 10x cheaper than AWS

#237

Post by FireBeyond »

xienze wrote: post #192
vl wrote: post #190

Earlier quoted context omitted.

I have a hilarious story about this from Google: I wanted second 30" monitor, so I filed a ticket. They sent me long email listing reasons why I shouldn't get a second monitor, including (numbers are approximate, employee count from 2013 or so) "If every googler gets an extra monitor, in a year it would be equivalent to driving Toyota Camry for 18000 miles." I'm thinking "this can't possible be right", so I spend som…

> They sent me long email listing reasons why I shouldn't get a second monitor I had no idea Google was so cheap. Gourmet breakfast, lunch, and dinner every day? No problem. A couple hundred bucks for a second monitor? Uh... it's not about the money, we're, uh, concerned about the environment.

> Gourmet breakfast, lunch, and dinner every day? No problem.

These eke a few more hours of work out of you per day.

> A couple hundred bucks for a second monitor?

Arguments about productivity aside (I agree, more productive), these don't.

Re: Building your own deep learning computer is 10x cheaper than AWS

#238

Post by cozzyd »

PascLeRasc wrote: post #199
nabla9 wrote: post #122

Earlier quoted context omitted.

That's very passive aggressive way to deal with it. If that's your only option you are really a cubicle slave in corporate hell. In my opinion it's better to escalate upwards with proposals and not back down easily. You just have to frame it correctly and use right names and terms. * Usually big companies understand the concept of "a lab" that has infrastructure managed outside the corporate IT. Once you fight the ha…

I wanted to put an Ubuntu partition on my work PC for Python deep learning work, as I'm significantly faster and happier on it. When I mentioned it to the sysadmin, he said "I'm not allowing that. Linux is like Wikipedia, any idiot can contribute to it. Windows is made by professionals so it has to be better."

Is that what people learn in Windows sysadmin school?

Re: Building your own deep learning computer is 10x cheaper than AWS

#239

Post by mmt »

poulsbohemian wrote: post #230
dagw wrote: post #29

You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)

Could be very interesting when we have another economic downturn to see if attitudes on this change. It certainly seems more cost-effective to run ones own technical operations rather than offloading onto AWS / Google / Microsoft.

It will only matter if the infrastructure is a large part of their cost of technical operations (as opposed to labor) or, more importantly, overall cost of operations.

In another thread on here (unfortunately I can't recall which), an executive shared a sentiment along the lines of "I don't care if it's 1% or 0.1% of my overall budget".

Perhaps 0.9% would become more significant during a downturn, especially for startups if VC money dries up.

Re: Building your own deep learning computer is 10x cheaper than AWS

#240

Post by mmt »

peter303 wrote: post #17

Did the Stanford grad include that his and other grad students time is worth at least $100 an hour on the open market, i.e. what a Stanford CS grad would make in Silicon Valley (cest moi)?

This is only applicable if they can actually sell very small increments of their time for that amount of money, which I strongly suspect they can't. There have been numerous threads here on HN about the difficulties of finding full-price work that isn't full time.

Even then, the $2k difference between the cheapest pre-built the article references and the DIY version would be 20 hours at $100/hr. Half a workweek to build one PC seems excessive.

Post reply on HN