Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building your own deep learning computer is 10x cheaper than AWS

medium.com

Post reply on HN
Print view View original

211–220 of 269 posts

Re: Building your own deep learning computer is 10x cheaper than AWS

#211

Post by walshemj »

flatb wrote: post #176

Can anybody fill me in on why people these days are spending almost the same money for a last-gen, medium-high end, 12 core, 24 thread CPU and a motherboard? What features would an almost $400 motherboard have that could justify that price?

The thread ripper chip sets x399 are expensive and also if you want to put 4x GPU's not all MB support this and the power required to run this all.

Baring in mind spending $400 on a mb pales when you are taking about a $1800 dollar CPU and 4x high end graphic's cards

Re: Building your own deep learning computer is 10x cheaper than AWS

#212

Post by marme »

unbearded wrote: post #165
dagw wrote: post #29

You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)

Unfortunately, the fight goes even further than that when you go against the cloud. Last week I was in an event with the CTOs of many of the hottest startups in America. It was shocking how much money is wasted on the cloud because inefficiencies and they simply don't care how much it costs. I guess since they are not wasting their own money, they can always come up with the same excuse: developers are more expensive…

If you buy equipment you can only deduct its depreciation costs every year. If you pay for services you can deduct the full cost from your taxes.

Re: Building your own deep learning computer is 10x cheaper than AWS

#213

Post by tomc1985 »

scurvy wrote: post #151
nabla9 wrote: post #122

Earlier quoted context omitted.

That's very passive aggressive way to deal with it. If that's your only option you are really a cubicle slave in corporate hell. In my opinion it's better to escalate upwards with proposals and not back down easily. You just have to frame it correctly and use right names and terms. * Usually big companies understand the concept of "a lab" that has infrastructure managed outside the corporate IT. Once you fight the ha…

In my experience, the blind drive to the cloud comes from above. It's someone's pet project and you're either on-board or not. I won't go as far as calling it yet another cyclical IT phase, but it has all the hallmarks of one.

There are armies of salespeople brainwashing executives about "the cloud". At my last job it only took a few years before every single C-level was spouting off recycled cloud propaganda

Re: Building your own deep learning computer is 10x cheaper than AWS

#214

Post by walshemj »

d--b wrote: post #6

Of course building is cheaper than AWS... It also means you need to build the computer, maintain it, upgrade it, store it somewhere, connect it to the internet, and so on. And you can't just scale it if you need to. I mean, the cloud's here for a reason...

You can always just hook up a new rack to expand on prem HPC

Re: Building your own deep learning computer is 10x cheaper than AWS

#215

Post by mmt »

lotyrin wrote: post #177
scurvy wrote: post #151

Earlier quoted context omitted.

In my experience, the blind drive to the cloud comes from above. It's someone's pet project and you're either on-board or not. I won't go as far as calling it yet another cyclical IT phase, but it has all the hallmarks of one.

It has to be, even with as irrational as the market can be I don't see people sustaining this kind of spend on public clouds -- the cultural barriers and perverse incentives that prevent effective use of private / hybrid cloud have to erode eventually.

> have to erode eventually

That's not borne out by history, especially in the face of "enterprise" hardware, software, and support, which, in many large companies, is being replaced by cloud (at lower cost!).

For smaller companies, it may be a different story, especially at the next economic downturn, especially if VC money becomes scarce enough for long enough.

Re: Building your own deep learning computer is 10x cheaper than AWS

#216

Post by karmajunkie »

marme wrote: post #212
unbearded wrote: post #165

Earlier quoted context omitted.

Unfortunately, the fight goes even further than that when you go against the cloud. Last week I was in an event with the CTOs of many of the hottest startups in America. It was shocking how much money is wasted on the cloud because inefficiencies and they simply don't care how much it costs. I guess since they are not wasting their own money, they can always come up with the same excuse: developers are more expensive…

If you buy equipment you can only deduct its depreciation costs every year. If you pay for services you can deduct the full cost from your taxes.

You can typically take the full depreciation in the year you purchased the equipment under IRS Section 179, up to a limit which varies depending on which way the wind is blowing in Congress. For 2018 the limit is $1MM. Whether or not its more beneficial to you to take the depreciation over time is a question for your accountant—technically if you later sell the equipment you're supposed to recapture the revenue from the sale for tax purposes.

Re: Building your own deep learning computer is 10x cheaper than AWS

#217

Post by paulmd »

flatb wrote: post #176

Can anybody fill me in on why people these days are spending almost the same money for a last-gen, medium-high end, 12 core, 24 thread CPU and a motherboard? What features would an almost $400 motherboard have that could justify that price?

Threadripper is actually not even that great here, it's NUMA (half of the lanes are attached to each die) and if you just want NUMA you can do a dual-socket E5-2650 system for like half the price. Same logic applies to TR here:

https://www.servethehome.com/how-intel-skylake-sp-changes-im...

You're not using the CPU for anything, there is no reason to have a beefy 12-core CPU there. If you really want Threadripper, the 8-core version is fine.

Again, personally I'd look at X79 boards, since they are pretty cheap and you can do up to 4 GPUs off a single root, depending on the board. There are new-production boards available from China on eBay, see ex: "Runing X79Z". Figure about $150 for the mobo, about $100 for the processor, and then you can stack in up to 128 GB of RAM, including ECC RDIMMs, which runs about $50 per 16 GB.

There are some Z170/Z270 boards like the "Supercarrier" from Asrock that include PLX chips to multiplex your x16 lanes up to 4 16x slots (does not increase bandwidth, just allows more GPUs to share it at full speed!). They also will not support ECC (you need a C-series chipset for that, which run >$200). So far most OEMs have been avoiding putting out high-end boards for Z370 because they know Z390 is right around the corner so there is currently no SuperCarrier available for the 8000 series.

Re: Building your own deep learning computer is 10x cheaper than AWS

#218

Post by rever »

mrguyorama wrote: post #185
titanomachy wrote: post #161

Earlier quoted context omitted.

Normal food can be even cheaper. You could probably indefinitely avoid starvation with dried rice and beans, oil, and a vitamin supplement for $2/day (in NA), without a big upfront cost. Poverty is complicated.

You've ignored the cost of the home required to cook those things in, which is really the whole point. (not directing at you, more explaining to those who didn't already know)

That recent Boston(?)-based completely automated rice bowl startup plus drone delivery equals solved hunger

Re: Building your own deep learning computer is 10x cheaper than AWS

#219

Post by mangymutt »

marme wrote: post #212
unbearded wrote: post #165

Earlier quoted context omitted.

Unfortunately, the fight goes even further than that when you go against the cloud. Last week I was in an event with the CTOs of many of the hottest startups in America. It was shocking how much money is wasted on the cloud because inefficiencies and they simply don't care how much it costs. I guess since they are not wasting their own money, they can always come up with the same excuse: developers are more expensive…

If you buy equipment you can only deduct its depreciation costs every year. If you pay for services you can deduct the full cost from your taxes.

Anyone who thinks spending $10 instead of $1, so that you can book the expense (presumably for a tax write-off which might save you 30%... MAYBE?) needs to stay away from Finances.

Re: Building your own deep learning computer is 10x cheaper than AWS

#220

Post by walshemj »

dagw wrote: post #56
emptyfile wrote: post #51

Earlier quoted context omitted.

Is it really, in America? Amazing. This is not the normal human experience at all. Raw food are usually cheaper than processed ones, anywhere in the world.

Where in the world are you? It's my normal human experience from both the UK and Sweden. Of course the stuff you make from scratch will often have a higher quality (or at least be made from higher quality ingredients) and if you want to buy similar or better quality then it will cost more. But the cheapest chocolate chip cookies you can find in the food store will be cheaper than buying ingredients and baking.

It will be cheaper if you buy in bulk the local Tescos to me in the UK sells rice in 10Kg bags which is several times cheaper than boil in the bag

And I am sure the local Asian cash and carry's sell even bigger bags even cheaper.

Post reply on HN