Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building your own deep learning computer is 10x cheaper than AWS

medium.com

Post reply on HN
Print view View original

151–160 of 269 posts

Re: Building your own deep learning computer is 10x cheaper than AWS

#151

Post by scurvy »

nabla9 wrote: post #122
dagw wrote: post #29

You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)

That's very passive aggressive way to deal with it. If that's your only option you are really a cubicle slave in corporate hell. In my opinion it's better to escalate upwards with proposals and not back down easily. You just have to frame it correctly and use right names and terms. * Usually big companies understand the concept of "a lab" that has infrastructure managed outside the corporate IT. Once you fight the ha…

In my experience, the blind drive to the cloud comes from above. It's someone's pet project and you're either on-board or not.

I won't go as far as calling it yet another cyclical IT phase, but it has all the hallmarks of one.

Re: Building your own deep learning computer is 10x cheaper than AWS

#152

Post by scottlegrand2 »

dagw wrote: post #89
badpun wrote: post #82

Earlier quoted context omitted.

So, whatever bakery paid for ingredients, wages and owner's profit is still less that what you'd just pay for just the ingredients? What are they putting into these cookies, sawdust?

What are they putting into these cookies Cheap palm oil and other vegetable fats instead of butter, corn syrup instead of cane sugar, much lower quality chocolate than is generally available in grocery stores, various chemical stabilizers instead of eggs, artificial vanilla flavoring instead of real vanilla and so on. Plus they obviously get everything at a massive discount since they're buying everything by the ton…

You might want to ask why those cheap ingredients are cheap and you will find a bunch of farmers getting huge subsidies from the taxpayers to grow otherwise unprofitable crops.

Kind of like how if you remove all the subsidies given to fossil fuel producers, some forms of renewable energy suddenly become much more competitive.

So when you add a whole cost center for IT, the inefficiencies of the way they operate suddenly make the cloud efficient.

Re: Building your own deep learning computer is 10x cheaper than AWS

#153

Post by jimmy1 »

ConcernedCoder wrote: post #131
dagw wrote: post #29

You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)

Innovation being sidelined in favor of status quo and 'cover-my-ass' risk management highlights exactly the reason I won't work for a bureaucratic corporation...

Money^, benefits^^, ample PTO^^^, and a 8-5 workday allowing plenty of time for the important things in life like my family are exactly the reasons I do.

^: You know, like real money, not 90k plus 2.5% of something that will probably not exist next year

^^: You know, like real benefits such as low-cost, low-deduct. health insurance, matching 401k, etc.

^^^: As in, real, actual PTO, not this "unlimited" bullshit some startups and now sadly companies are replicating. If you decide to quit and can't "cash out" your PTO, you don't have real PTO.

Re: Building your own deep learning computer is 10x cheaper than AWS

#154

Post by wetpaws »

solatic wrote: post #109
VHRanger wrote: post #70

Earlier quoted context omitted.

They seriously can't buy a graphics card and slap it in the PCIe slot?

You're neglecting management costs. IT teams don't buy hardware with corporate credit cards, they have to work through pre-existing requisition processes that properly budget for the hardware, make sure support contracts are in place, etc. You have to migrate whatever workload off the server where you installed the GPU (politically problematic since Murphy promises you that your users will be connecting to the server…

Why I as developer should worry about this? It's their job.

Re: Building your own deep learning computer is 10x cheaper than AWS

#155

Post by w8rbt »

dagw wrote: post #29

You're forgetting the cost of fighting IT in a bureaucratic corporation to get them to let you buy/run non-standard hardware Much easier to spend huge amounts of money of Azure/AWS and politely tell them it's their own fucking fault when they complain about the costs. (what me? no I'm not bitter, why do you ask?)

Yep. No one telling me no anymore and I can write Lambas to replace cron jobs, use RDS to replace DBs, use S3 and Glacier to replace Storage, etc. Fargate is awesome too. No gate keepers nor bureaucracy just code and git repos. That's why AWS is so awesome.

And, I can show exactly how much everything costs. As well as work to reduce those costs.

AWS added to a small, skilled dev team is a huge multiplier.

Re: Building your own deep learning computer is 10x cheaper than AWS

#156

Post by bunderbunder »

davnicwil wrote: post #93
maltalex wrote: post #24

I don't see what's surprising here. This is the ski rental problem [0]. Would it surprise anyone to learn that renting a car is more expensive in the long run compared to buying one? This is the same thing, only the time scales are different. [0]: https://en.wikipedia.org/wiki/Ski_rental_problem

But the ski rental problem as described there misses the same factors as the AWS example: it does not account for the additional costs of ownership vs renting. The analogy doesn't quite map for all aspects so I'll stick with the ski example to show it's not specific to cloud computing: Owning skis has a load of associated costs ranging from simple to more complex: storage, maintenance, carrying them to and from the a…

Concrete example: I own skis. But if I'm going on a ski vacation instead of a day trip, I rent them. That decision has nothing to do with the cost to rent skis vs. what they sell for, and everything to do with the fact that I can rent skis for about the same as what it would cost me to transport my own skis and boots to Breckenridge and back.

I suspect that there is something like that (only not at all) going on with machine learning kit: If you've got exactly one person training models, and they're doing it all the time, then yeah, it's cheaper to own your own hardware. If you've got several people all trying to share the training rig, then you start having to worry about all the costs associated with scheduling time, delays if someone else got their first (especially if they're doing deep learning and will be monopolizing the server for upwards of a week), etc. Once you factor all those full-lifecycle costs in, the cloud might start to look relatively cheaper.

Re: Building your own deep learning computer is 10x cheaper than AWS

#157

Post by w8rbt »

I have seen sys admins stand-up a bunch of EC2's in AWS and install PostGres and Docker on them (because the dev's said they need a DB and a docker server). They don't get the services model (use RDS and ECS). Sys admins have to change. Orgs can't afford this cost nor be slowed down by this 1990s mindset.

Standing up a bunch of EC2's in AWS is just a horrible idea and an expensive one as well. It also moves all of the on-prem problems (patching, backups, access, sys admins as gatekeepers, etc.) to the cloud. It's the absolutely wrong way to use AWS.

So stop sys admins from doing that as soon as you notice. Teach them about the services and how, when used properly, the services are a real multiplier that frees everyone up to do other, more important things rather than baby sitting hundreds of servers.

Re: Building your own deep learning computer is 10x cheaper than AWS

#158

Post by _hyn3 »

desdiv wrote: post #81
swebs wrote: post #66

Earlier quoted context omitted.

I've seen just the opposite. A VPS from a cloud provider is often 2-3x the cost of an equivalent Linode.

I think your pricing information might be slightly out of date. AWS Lightsail's $5 plan is equivalent or better than Linode's $5 plan. Plus AWS Lightsail has an even more barebones plan at $3.5 per month. Two years ago you would be perfectly correct. The cheapest AWS option back then would be a micro EC2 instance at around $20 per month IIRC.

Lightsail is basically a toy to compete with Vultr/DO/OVH/etc. You can't even rename a server or change its metadata. You can't resize a server (except for taking a snapshot and building a new server). Essentially, you can't make any changes to a server and there are zero tools for organizing, tagging, or even adding a description field. Of course, you cannot manage Lightsail instances using the AWS console (or even see them). All of the powerful AWS tools are invisible to you as well, and your entire Lightsail infrastructure is invisible from within AWS itself (except for the bill).

Lightsail is really great for personal sandbox servers, or for moving to AWS on the cheap, but it's not even as capable as its worst competitor (and certainly cuts out a lot of features compared to real AWS.)

The worst part is probably the surprising and seemingly minor factor of being able to simply rename or resize a server. This makes using it in a team or production environment surprisingly difficult.

With that all said, I still find Lightsail very useful for small prototyping and other fast jobs (or a Lambda replacement -- it's way cheaper than running any reasonably busy Lambda function), but I can't see using it for real production usage. Your best bet for real production usage, or something that will someday move into production, would be probably vultr/DO/linode. (OVH really wants to be great, and it should be because the hardware is really great, but the dashboard and billing are so bizarrely bad..)

AWS is good when you have a virtually unlimited budget or don't mind running up the bill while you're getting rolling, but getting locked into AWS will mean that you are stuck with it forever. It's very hard to migrate away if you start using a vendor-specific thing like DynamoDB (to be fair, Google Firebase has exactly the same issue; Instances are basically fungible, but proprietary data stores are not)

Re: Building your own deep learning computer is 10x cheaper than AWS

#159

Post by malux85 »

w8rbt wrote: post #157

I have seen sys admins stand-up a bunch of EC2's in AWS and install PostGres and Docker on them (because the dev's said they need a DB and a docker server). They don't get the services model (use RDS and ECS). Sys admins have to change. Orgs can't afford this cost nor be slowed down by this 1990s mindset. Standing up a bunch of EC2's in AWS is just a horrible idea and an expensive one as well. It also moves all of th…

Yeah that's nice and all, but then you have total and complete vendor lock-in.

I decided to do the EC2 thing when I built one of my products, knowing that I couldn't have vendor lock in - and that decision was critical to the survival of my company when:

1) A customer wanted to run on azure. We would have lost a 2.5 million pound contract if we couldn't do that

2) Another customer wanted on-prem solution, we would have lost a 55 million USD contract if we were vendor locked to AWS

so sometimes it makes sense

Re: Building your own deep learning computer is 10x cheaper than AWS

#160

Post by gnufx »

By coincidence I just posted https://news.ycombinator.com/item?id=18066472 about the expense of AWS et al for NASA's HPC work. (Deep learning, "big data" et al are, or should be, basically using HPC and general research computing techniques, although the NASA load seems mostly engineering simulation.)
Post reply on HN