Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Self-Hosting Still Pays

servethehome.com

Post reply on HN
Print view View original

31–40 of 343 posts

Re: Self-Hosting Still Pays

#31

Post by sabalaba »

Always love Patrick's posts. Lambda has built a business on moving people from the cloud to on-prem and reducing the cost of cloud AI infrastructure. As some of the other comments have pointed out, for machine learning applications with GPUs, it almost always makes sense to run your workloads on prem, STH is showing that it can also true for traditional CPU workloads.

For example, we're able to provide GPU instances (https://lambdalabs.com/service/gpu-cloud) that are half the hourly cost of AWS hourly on-demand pricing. How? Because there are huge markups on clouds services.

We've done such extensive benchmarking and TCO analysis and the jury is out: it's simply less expensive to run on-prem. You're just paying for convenience when using a cloud. GPU or otherwise.

Sources:

- https://lambdalabs.com/gpu-benchmarks

- https://lambdalabs.com/blog/hyperplane-16-infiniband-cluster...

Re: Self-Hosting Still Pays

#32

Post by Patrick-STH »

bpodgursky wrote: post #22

Even ignoring labor, none of these analyses ever consider BC/DR or backup costs. S3 for backups is the single biggest ease-of-mind feature AWS provides, IMO. I do not want all my data sitting in a single colocation facility without backups. If I run a database on RDS, maybe it costs more than a bare-metal server, but I can replicate the automatic hourly backups to as many regions as I want. Incredible peace of mind o…

Those are included in our colo costs with a second data center where we have a smaller DR cluster.

Re: Self-Hosting Still Pays

#33

Post by munk-a »

anonfornoreason wrote: post #27
Rantenki wrote: post #14

Earlier quoted context omitted.

That's false equivalence. If your instance dies, you don't need to drive over to the colo, plug into a KVM, diagnose, and then pull and re-rack a new server. They do all that stuff, and it's worth the $$$. Also, you'll never get into a situation on AWS where you have a 3 day lead-time for a replacement GBIC, or NVMe drive, or seomthing, while your customers scream and bail out for a competitor that isn't down. Yes, y…

I manage a rack of 12 servers, I have 110 hard drives in that rack. I've been managing this rack for a decade. I occasionally go to the datacenter to swap out a machine that is end of life, and I go every 3-6 months to swap out a failing drive or two. I'd estimate I spend Obvious anecdata here, but it's not hard to run a redundant physical infrastructure that is low maintenance. Obviously bad luck is a thing, but wit…

That's true - I was comparing managing a server rack in the office building against AWS - not comparing the relatively light hardware interaction needs when it comes to co-location.

Re: Self-Hosting Still Pays

#34

Post by jeffbee »

I have a feeling you could host that site on a PC at home for even less. It looks like their forum gets < 100 posts per day. I can't imagine they're hitting much more than 1 QPS.

Re: Self-Hosting Still Pays

#35

Post by Patrick-STH »

munk-a wrote: post #3

Wait - this article appears to compare hardware costs of self-hosting against the total cost of using AWS. It's missing the most expensive component (labour) which just so happens to be extremely expensive to scale.

Labor is included and discussed both for my time as well as remote hands we pay for. It is very low at this point.

Re: Self-Hosting Still Pays

#36

Post by Rantenki »

z92 wrote: post #16
Rantenki wrote: post #10

TIL: AWS is only expensive if your time has no value. Article neglected to add the cost of labour for setup (edit: aside of racking), maintenance (security updates on infrastructure, etc), and disaster response. It also missed the cost of downtime based on projected MTTR for a failure. The word "fail" only appears once in the page, and not in regards to recovery, only preventative replacement. It's absolutely true th…

He did, Quote : We are going to say we used four hours of labor. This includes drive time to the primary data center. Since it is far away (18-20 minute drive), we actually did not go there for several quarters. So over 32 months, we had budgeted $640 for remote hands. We effectively either paid $160/ hr or paid less than that even rounding up to four hours.

Read those figures again. They're such an underestimate as to be laughable, and apply only to the physical racking efforts.

There is no way in the world that the only effort taken related to infrastructure is entirely encapsulated in ~ $200 per year.

Re: Self-Hosting Still Pays

#37

Post by WhatIsDukkha »

KingMachiavelli wrote: post #20
shiftpgdn wrote: post #4

You don't even need to buy a new EPYC or Xeon Scalable system. A 20 core Xeon v3/v4 system with 128gb of ram can be had off lease for $1000. Psychz.net provides 1u colocation with 1/Gbs link (10/Gbs burst) for $80/mo. Equivalent EC2 is $300-600/month with no better reliability. Plus you have to deal with noisy neighbors and bandwidth overages. Build your baseline usage in colo and burst into AWS unless you like buyin…

> No better reliability? Pretty sure EC2 instances and EBS volumes have a lot more redundency than a single server. You really need two colocated servers to replace a single EC2 instance. Still probably cheaper but also a larger time investment. If the difference between AWS vs colocation is an additional FTE then AWS is cheaper.

I'm always nonplussed by the "additional FTE" argument, its obscenely overestimated -

"""We are going to say we used four hours of labor. This includes drive time to the primary data center. Since it is far away (18-20 minute drive), we actually did not go there for several quarters. So over 32 months, we had budgeted $640 for remote hands. We effectively either paid $160/ hr or paid less than that even rounding up to four hours. """

Software and systems configuration wise you aren't really going to be doing much different timewise then you would by doing loop de loops with AWS configuration stuff anyway. Tftp etc is just not that tough.

Re: Self-Hosting Still Pays

#38

Post by nikisweeting »

honkycat wrote: post #17

Good luck hiring the professionals needed to run a bunch of co-located servers in 2020.

Maybe this is my old-school sysadmin showing, but how is managing a rack server more difficult than managing an EC2 instance?

The hardware part is not difficult, if you've built a PC you can set up a rack server (it's arguably easier because they're built to assemble without needing tools), and the software is all Linux on both, no? Or do you mean hiring on-site engineers within driving distance of your rack servers is difficult?

Re: Self-Hosting Still Pays

#39

Post by Patrick-STH »

jedberg wrote: post #28

These calculations never account for capital overhead of having spares on hand. That cost is part of what you pay for on Amazon. If the hardware breaks on Amazon, you just switch hardware. If hardware breaks in your datacenter, it's an emergency and you better have spare capacity or extra hardware laying around.

We include in our "Hardware Costs" having extra node and spares. For example, we keep a full spare node in the DC as well as spares.

Re: Self-Hosting Still Pays

#40

Post by Patrick-STH »

Rantenki wrote: post #36
z92 wrote: post #16

Earlier quoted context omitted.

He did, Quote : We are going to say we used four hours of labor. This includes drive time to the primary data center. Since it is far away (18-20 minute drive), we actually did not go there for several quarters. So over 32 months, we had budgeted $640 for remote hands. We effectively either paid $160/ hr or paid less than that even rounding up to four hours.

Read those figures again. They're such an underestimate as to be laughable, and apply only to the physical racking efforts. There is no way in the world that the only effort taken related to infrastructure is entirely encapsulated in ~ $200 per year.

100%. We had 11 months where nobody touched the racks as an example and the visit 11 months later was the one to physically get a tool I had left there. I mentioned I even track drive time.
Post reply on HN