Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building your own deep learning computer is 10x cheaper than AWS

medium.com

Post reply on HN
Print view View original

51–60 of 269 posts

Re: Building your own deep learning computer is 10x cheaper than AWS

#51

Post by emptyfile »

amelius wrote: post #16
d--b wrote: post #6

Of course building is cheaper than AWS... It also means you need to build the computer, maintain it, upgrade it, store it somewhere, connect it to the internet, and so on. And you can't just scale it if you need to. I mean, the cloud's here for a reason...

> Of course building is cheaper than AWS... Why "of course"? Last time I checked baking my own cookies is more expensive than just buying a bag at Walmart.

Is it really, in America? Amazing.

This is not the normal human experience at all. Raw food are usually cheaper than processed ones, anywhere in the world.

Re: Building your own deep learning computer is 10x cheaper than AWS

#52

Post by yonkshi »

emptyfile wrote: post #51
amelius wrote: post #16

Earlier quoted context omitted.

> Of course building is cheaper than AWS... Why "of course"? Last time I checked baking my own cookies is more expensive than just buying a bag at Walmart.

Is it really, in America? Amazing. This is not the normal human experience at all. Raw food are usually cheaper than processed ones, anywhere in the world.

Not necessarily. E.g. making Ramen noodles, much cheaper to go to a ramen shop than to attempt to make it yourself. Economy of scale.

Re: Building your own deep learning computer is 10x cheaper than AWS

#53

Post by hannasanarion »

denzil_correa wrote: post #47
maltalex wrote: post #24

I don't see what's surprising here. This is the ski rental problem [0]. Would it surprise anyone to learn that renting a car is more expensive in the long run compared to buying one? This is the same thing, only the time scales are different. [0]: https://en.wikipedia.org/wiki/Ski_rental_problem

So, what's the optimal solution according to the Ski Rental problem?

It depends on your needs. If you are absolutely certain that you are going to rent skis so often that you will spend more then buying them (plus the costs of storage, transport, and maintenance), then buying is the right choice.

Re: Building your own deep learning computer is 10x cheaper than AWS

#54

Post by code4tee »

Purely on hardware yes it’s no secret that AWS is more, quite a bit more, than just buying/building the machine and plugging it in. That’s for any servers no just “deep learning” servers.

Of course you’re also paying for everything else AWS brings and the ability to spin up/down on demand with nearly unlimited scalability which is hardly “free.” AWS is also a very profitable business for Amazon so they’re making good margin too on most of their pricing.

Re: Building your own deep learning computer is 10x cheaper than AWS

#55

Post by hmage »

ofrzeta wrote: post #15
psergeant wrote: post #4

> Nvidia contractually prohibits the use of GeForce and Titan cards in datacenters. So Amazon and other providers have to use the $8,500 datacenter version of the GPUs, and they have to charge a lot for renting it.

I wondered how it is possible that they can restrict the use of hardware after it is bought. After reading the article I learned that they key is the license to the drivers. So I guess in theory it would be possible for AWS to develop their own (or enhance open source) drivers. On the other hand they would spoil the business relationship with Nvidia and have to do without any discounts.

The problem is that there's already open-source nvidia driver -- https://nouveau.freedesktop.org -- but it's so feature incomplete that it's good only for hobby. You can't run CUDA on it, you can't run any recent games on it, you can run TensorFlow on it, you can't run PyTorch on it, etc etc -- it is essentially useless and has been so for many years.

Re: Building your own deep learning computer is 10x cheaper than AWS

#56

Post by dagw »

emptyfile wrote: post #51
amelius wrote: post #16

Earlier quoted context omitted.

> Of course building is cheaper than AWS... Why "of course"? Last time I checked baking my own cookies is more expensive than just buying a bag at Walmart.

Is it really, in America? Amazing. This is not the normal human experience at all. Raw food are usually cheaper than processed ones, anywhere in the world.

Where in the world are you? It's my normal human experience from both the UK and Sweden. Of course the stuff you make from scratch will often have a higher quality (or at least be made from higher quality ingredients) and if you want to buy similar or better quality then it will cost more. But the cheapest chocolate chip cookies you can find in the food store will be cheaper than buying ingredients and baking.

Re: Building your own deep learning computer is 10x cheaper than AWS

#57

Post by gnur »

> Even when you shut your machine down, you still have to pay storage for the machine at $0.10 per GB per month, so I got charged a couple hundred dollars / month just to keep my data around.

Curious how it relates to sticking only a single terabyte SSD in the machine. As a couple hundred dollars per month should relate to a couple terabytes.

Re: Building your own deep learning computer is 10x cheaper than AWS

#58

Post by dagw »

v_lisivka wrote: post #41
soneil wrote: post #30

Earlier quoted context omitted.

I think the analogy holds. The numbers in the article assume constant usage (there is a nod to the cost of static storage, but all the math is done on $3/hr gpu usage, not $0.1/mo/GB storage). So if your usage is 4 hours per week, 50 weeks a year, you hit $600/year of gpu. If your usage is 24 hours a day, you hit $27k. So it's not a clear-cut "buying metal is cheaper" - you have to decide how your usage amortizes. As…

For DL time is measured in years*GPU instead of hours.

Only for niche problems run into by huge companies and research groups. Most 'normal' problems encountered by 'normal' companies don't need anywhere near that much training.

Re: Building your own deep learning computer is 10x cheaper than AWS

#59

Post by aranw »

montenegrohugo wrote: post #2

I really want to believe this. Of course the numbers given depend on very frequent use of your machine, but still. One would imagine that building a datacenter at scale and only renting when you actually are training a model to be much cheaper, but the reality appears to be not so. So where does the money go? Three places: - AWS/Google/Whoever-you're-renting-from obviously get a cut - Inefficiencies in the process (t…

I'm guessing the costs around NVIDIA cards is why Google are pushing there own TPU's and building custom chips and hardware?

Re: Building your own deep learning computer is 10x cheaper than AWS

#60

Post by rcarmo »

gnur wrote: post #57

> Even when you shut your machine down, you still have to pay storage for the machine at $0.10 per GB per month, so I got charged a couple hundred dollars / month just to keep my data around. Curious how it relates to sticking only a single terabyte SSD in the machine. As a couple hundred dollars per month should relate to a couple terabytes.

Yeah, that didn't add up for me too. Storage for a single machine shouldn't add up to hundreds of dollars a month unlesa you're not really trying to manage your data (by culling datasets, using compression, etc.)
Post reply on HN