Building your own deep learning computer is 10x cheaper than AWS • Hacker News Board

Re: Building your own deep learning computer is 10x cheaper than AWS

#121

Post by mastax » Tue, Sep 25, 2018, 1:59 PM UTC

screye wrote: post #63

psergeant wrote: post #4

> Nvidia contractually prohibits the use of GeForce and Titan cards in datacenters. So Amazon and other providers have to use the $8,500 datacenter version of the GPUs, and they have to charge a lot for renting it.

Is Nvidia doing this only because they are a monopoly in the space and practically extorting here ? Or are there any genuine costs associated with data center gpu models ?

It's market segmentation. AMD does the same thing. $600 consumer card becomes $4000 FirePro card with no hardware changes.

Re: Building your own deep learning computer is 10x cheaper than AWS

#122

Post by nabla9 » Tue, Sep 25, 2018, 2:06 PM UTC

dagw wrote: post #29

You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)

That's very passive aggressive way to deal with it. If that's your only option you are really a cubicle slave in corporate hell.

In my opinion it's better to escalate upwards with proposals and not back down easily. You just have to frame it correctly and use right names and terms.

* Usually big companies understand the concept of "a lab" that has infrastructure managed outside the corporate IT. Once you fight the hard fight, you get your own corner and are left alone to do your job and can gradually grow it for other things.

* Asking forgiveness works even for large companies. Sometimes someone is not happy and you 'are in trouble' but not really. You just have to have a personality that does not care if somebody gets mad at you sometimes.

Re: Building your own deep learning computer is 10x cheaper than AWS

#123

Post by meddlepal » Tue, Sep 25, 2018, 2:06 PM UTC

askmike wrote: post #86

ethbro wrote: post #79

Earlier quoted context omitted.

Respectfully, those companies' cloud architects suck. If someone goes to the trouble to migrate onto cloud, and then replicates pre-devops workflows... wow.

This has less to do with cloud vs own hardware and more about how the company is structured. I've worked in companies before with an ops department: a few people responsible for managing all the cloud servers. All the devs (like me at the time) work locally and have access to an isolated dev/uat environment provided by that team. That team had most of the show automated, they weren't really provisioning any machines…

> I guess this is what you are calling a pre-devops workflow? In a lot of fields not all devs are allowed to see/touch the complete production environment. Not everyone can go the netflix way of "everyone pushes to production and we'll just fix it when it breaks"

Unwillingness shouldn't be confused with ability. Most companies can do this if they're not handling PII/PHI. It takes investment in smart people and time but most companies besides pure software companies see tech as a cost center and avoid investing in better infrastructure and platform systems.

Re: Building your own deep learning computer is 10x cheaper than AWS

#124

Post by whatshisface » Tue, Sep 25, 2018, 2:06 PM UTC

supernovae wrote: post #114

ethbro wrote: post #79

Earlier quoted context omitted.

Respectfully, those companies' cloud architects suck. If someone goes to the trouble to migrate onto cloud, and then replicates pre-devops workflows... wow.

Not every company is a single page webapp and simple service portfolio... I work at a place with 3,000+ developers and over 700 applications - there is no way in hell our cloud portfolio would have any standards if we didn't have a robust operations/engineering team making it work. Sometimes, even when you adopt the cloud, you realize that your operations model is even more important and there is nothing wrong with t…

The parent's point was that if the ops/deployment engineering team is unresponsive to the needs of developers, it may end up being better to run with no standards in cowboy-mode. If they ops team is extremely fast and highly skilled, they will be a boon full stop. If they are unskilled and politically obstructive, they will be a curse, full stop.

Re: Building your own deep learning computer is 10x cheaper than AWS

#125

Post by quantumhobbit » Tue, Sep 25, 2018, 2:11 PM UTC

drewg123 wrote: post #85

VHRanger wrote: post #70

Earlier quoted context omitted.

They seriously can't buy a graphics card and slap it in the PCIe slot?

When I worked at Google, I really missed the dual monitor setup I had had at my previous job. I asked my manager how to get a 2nd monitor. Apparently, since I had the larger monitor, I was not allowed to get a 2nd one without all kinds of hassle. I asked if I was allowed to just buy one from Amazon and plug it in, and I was told no. I finally just grabbed an older one that had been sitting in the hallway of the cube-…

At an old job I managed to snag a 30" monitor from a colleague who was moving to a different job. He managed to get it in the brief window that it was offered instead of 26" ones. When I announced that I was moving on, the vultures started circling over who would get the monitor.

The inventory sheet will never be reconciled over that monitor, unless it breaks.

Re: Building your own deep learning computer is 10x cheaper than AWS

#126

Post by dceddia » Tue, Sep 25, 2018, 2:12 PM UTC

maltalex wrote: post #24

I don't see what's surprising here. This is the ski rental problem [0]. Would it surprise anyone to learn that renting a car is more expensive in the long run compared to buying one? This is the same thing, only the time scales are different. [0]: https://en.wikipedia.org/wiki/Ski_rental_problem

TIL "ski rental problem". Makes sense that this would be an entire class of problem.

To me it's been the "should I buy an airplane" problem. (Basically, you have to rent a lot, > 50 hrs/year, before buying makes sense financially, and even then it might not work out very well unless you fly a ton)

Re: Building your own deep learning computer is 10x cheaper than AWS

#127

Post by dkobran » Tue, Sep 25, 2018, 2:18 PM UTC

NVIDIA is attempting to separate enterprise/datacenter and consumer chips to justify the cost disparity. Specifically, they're introducing memory, precision etc. limits which have major performance implications to GeForce and there's also the EULA which was been mentioned here. That said, everything AWS comes at a premium as they're making the case that on-demand scale outweighs the pain of management/CapEx. This premium is especially noticeable with more expensive gear like GPUs. At Paperspace (https://paperspace.com), we're doing everything we can to bring the cost down of cloud and in particular, the cost of delivering a GPU. Not all cloud providers are the same :)

Disclosure: I work on Paperspace

Re: Building your own deep learning computer is 10x cheaper than AWS

#128

Post by quantumhobbit » Tue, Sep 25, 2018, 2:22 PM UTC

Does this factor in the cost of lost productivity from having an awesome gaming pc always available?

Re: Building your own deep learning computer is 10x cheaper than AWS

#129

Post by throwaway2048 » Tue, Sep 25, 2018, 2:23 PM UTC

ethbro wrote: post #79

mlthoughts2018 wrote: post #45

Earlier quoted context omitted.

In most companies, AWS just becomes a new front-end to the same old IT bureaucracy, and dev teams are still disallowed from creating their own instances or EMR clusters or setting up new products like their own Redshift instance or ECR deployment solution.

Respectfully, those companies' cloud architects suck. If someone goes to the trouble to migrate onto cloud, and then replicates pre-devops workflows... wow.

why not cut out the middle man and just say those companies suck, and the cloud isn't going to fix it for you.

Re: Building your own deep learning computer is 10x cheaper than AWS

#130

Post by Const-me » Tue, Sep 25, 2018, 2:24 PM UTC

code4tee wrote: post #54

Purely on hardware yes it’s no secret that AWS is more, quite a bit more, than just buying/building the machine and plugging it in. That’s for any servers no just “deep learning” servers. Of course you’re also paying for everything else AWS brings and the ability to spin up/down on demand with nearly unlimited scalability which is hardly “free.” AWS is also a very profitable business for Amazon so they’re making good…

> AWS is more, quite a bit more, than just buying/building the machine and plugging it in.

Last time I looked, for mid-range AWS instances, purchase price was about 6-12 months of the rent. That’s assuming you buy comparable servers, i.e. Xeon, ECC RAM, etc…

For GPGPU servers however, purchase price is only 1-2 months of Amazon rent. Huge difference, despite 1080Ti is very comparable to P100, 1080Ti is slightly faster (10.6 TFlops versus 9.3), P100 has slightly more VRAM (12/16GB versus 11).