Live data from Hacker News

The cost of cloud

ptribble.blogspot.com

71–80 of 137 posts

Re: The cost of cloud

#71
post #25

These types of posts rarely measure operational cost. Anybody can buy infrastructure and stick it behind an API. But can you make it fault tolerant with high availability and low latency. Can you do all of that and _still_ beat AWS's costs? For the vast majority of customers the answer is no. I used to work at a medium sized company, and they saved millions by moving to the cloud, and gained much better availability/…

Not sure where you get “vast majority” from, but as an anecdote I worked for two companies that operated the platform teams at 10%-15% the cost before cloud (including headcount).

However, as I’ve eluded to in other threads of this kind; people don’t like to invest in their own tech, an ideal budget would have been around 15%-20% or cloud spend, we could have solved nearly everyone’s pain with that amount of money.

DR does mean paying more, but just so we’re clear: you have to do DR in cloud too, meaning read replicas of database instances and off-cloud backups which auto restore. If your platform costs more money then this is not a good thing and isn’t just baked into the normal cost.

Re: The cost of cloud

#72

I have a rule that is simple, effective but also quite rude: if you can't deliver and maintain a 500 instances large infrastructure, same uptime and all, at half the cost of AWS by yourself (1 person) in 3 months using only open source solutions basically you should not have an opinion about this. You are just rationalizing your incompetence on this particular subject. Sorry to be this blunt but I am simply tired of…

I maintained a datacenter with approx 1000 hypervisors with a very small team and took few weeks to start having production workloads. The effort to maintain hardware was quite little and it was hugely cheaper than any cloud service.

Having said that, your requirement is pretty absurd. Billions of people choose to own and maintain houses and cars and cook their own food because it's cheaper than the alternatives. Nobody expects them to be professional mechanics or cooks.

Re: The cost of cloud

#73
post #61

Earlier quoted context omitted.

I'm not even sure it's cheaper. At a previous employer we had approximately 150k of cloud expenses per annum. Bringing that in house would have eclipsed the cloud expenses on additional staffing costs alone.

What are you guys doing to have such a low cost? I have a small GKE cluster and a few databases and I’m well above that, I nearly hit that in a month! Maybe you don’t have read replicas of your databases? Do you take any traffic?

idk but "a small GKE cluster and a few databases" doesn't sound like it should cost a million per year.

Re: The cost of cloud

#74
post #10

I think the mistake is often made by comparing primitives. E.g. running my own RAID vs S3. Colo traffic vs AWS traffic. But what about comparing the whole ecosystem? Can you provide a self hosted granular access permission to your RAID? How hard is it to configure and maintain? Will your colo deflect a DDOS attack? When you run your own services, you have to reinvent so much it doesn’t seem to be worth it.

> Can you provide a self hosted granular access permission to your RAID? This is the second-level mistake engineers commonly make: The right questions isn't "Can you do X". Give engineers enough time and resources and they can usually come up with a solution to do X. The real question is "How much time and resources need to be invested to accomplish X at a satisfactory level?" And the third-level mistake is to assume…

You make a lot of claims without providing evidence.

Re: The cost of cloud

#75

I have a rule that is simple, effective but also quite rude: if you can't deliver and maintain a 500 instances large infrastructure, same uptime and all, at half the cost of AWS by yourself (1 person) in 3 months using only open source solutions basically you should not have an opinion about this. You are just rationalizing your incompetence on this particular subject. Sorry to be this blunt but I am simply tired of…

Which 500 instances? 500 ec2 “large” is like half of a rack… or is the point to engage opponent in unwinnable argument?

I think i can beat that argument pretty easy, 5 freebsd hosts with each 100 running jails...it's cheaper, uptime is something to discuss, but the data is not on someone elses computer.

The real cost of cloud is that nearly anyone think it's impossible to setup a infra for yourself....loosing systemadmin as a role in companys probably the biggest loss.

Re: The cost of cloud

#76

Hiring your own security guard is cheaper than paying an outsourcing firm. Hiring your own janitor is cheaper than paying an outsourcing firm. Building your own office is cheaper than renting one. Doing your taxes with pen and paper is cheaper than paying turbo tax. Making your own food is cheaper than eating out. Hiring a cook directly is cheaper than hiring sudexo. I could keep going. But sometimes it’s not just co…

Not to be dishonest towards the argument that you're making, but most of the examples that you give actually seem fairly straightforward.

I do my own taxes, but maybe that's just easier in Europe and is definitely easier for individuals. That said, there's no reason why a LibreOffice spreadsheet would be an insufficient solution for handling taxes and other things like that.

I also haven't eaten out in years, the closest to that was ordering some Wolt when hanging out with my friends pre-COVID, because they wanted to try some. Apart from that, it's all just home cooked meals for me and that's pretty great. It also seems to be working out great for the folks over at https://www.reddit.com/r/mealprep/top/?t=month

At work, the company that i work for have their own building and have their own support staff as well, which seems to be working out great for them.

Furthermore, there are plenty of on prem resources that are used and despite the disadvantage of lacking self-service in many cases, there's very little difference in configuring and running software for deployments, with something like Ansible and containers. Even moreso when you have to support clients that have their own particular data centers and on prem deployments, which might differ noticeably from public cloud offerings. That's even not thinking about things like compliance in regards to what data can be stored where.

Personally, i also have a homelab with some repurposed old computers with 200 GEs and value RAM, a few HDDs and WireGuard for working around NAT and exposing my sites to the world through a pretty cheap cloud VPS or two from https://www.time4vps.com/?affid=5294 (affiliate link, to make hosting cheaper if anyone else uses them). Of course, when i need 24/7 uptime, i do use their VPSes in a hybrid cloud setup, especially since my blog getting 30k views could be a bit taxing on a residental 4G LTE modem connection.

The argument about competencies, ecosystems, 3rd party offerings, outsourcing and so on is probably a valid for some, but not for me and not for many companies out there - too often you end up depending on SaaS solutions which vendor lock you and might cause you to spend unreasonable amounts of money, or will let you remain ignorant about how to actually manage the software that you're using, i think SaaSS (Service as a Software Substitute) is a relevant term here: https://www.gnu.org/philosophy/who-does-that-server-really-s...

That said, what works for me and even the company that employs me, won't work for others. And what works for others, won't work for me. This is all because of how different the circumstances of various people out there are: i cannot afford AWS, i cannot afford Azure, GCP and managed services for my own needs.

I currently pay 320 EUR for 6 cloud VPSes per year (and additional amounts for the occasional replacement HDD for my homelab), whereas others pay similar amounts for their cloud platforms of choice per month. For them, depending on their circumstances, it might be more cost effective to spend their time working and throw money at problems, whereas for me it's almost always more cost effective to learn the tech myself.

Similarly to how in Latvia you could hire a team of developers for what one developer would cost in the US. Companies have other factors to consider, of course, but this is just one example - the alternative (opportunity) costs of individuals.

Edit: Of course, some in the comments are talking about hundreds of VPSes/VMs/nodes and in my eyes, that's just an order of magnitude or two higher than what i'm talking about. I've seen plenty of companies in my country running their own data centers and there have been relatively few issues with those that i'm aware of. Something like Ansible and container clusters can scale pretty far!

The problems were more often caused by either mismanaged environments/deployments by developers/agencies who just didn't care about shipping sustainable software but cared more about getting paid and making their software someone else's problem, or making mistakes early in the development and not considering load testing and scalability of the systems as priorities. I'd argue that you can do bad engineering anywhere, though, be it on prem or in the cloud.

Re: The cost of cloud

#77

I have a rule that is simple, effective but also quite rude: if you can't deliver and maintain a 500 instances large infrastructure, same uptime and all, at half the cost of AWS by yourself (1 person) in 3 months using only open source solutions basically you should not have an opinion about this. You are just rationalizing your incompetence on this particular subject. Sorry to be this blunt but I am simply tired of…

The cost of three person months for a reasonably competent devops person is probably close to 50000$. Maybe a bit less for a cocky junior one that will make a mess and a bit more if you pay premium freelance rates for somebody less likely to botch the job. That pays for a lot of infrastructure. Not counting your own cost is a rookie mistake. And not realizing you really need 4-6 of these people to be able to get to y…

> And not realizing you really need 4-6 of these people to be able to get to your five nines is the second mistake (you need people on call 24x7 and when they are sick, over Christmas, etc).

If you need that kind of availability, you need to have people on call anyway to babysit your app. A good infrastructure (unless built to the minimal price point) will handle nearly all cases of hardware failure automatically, without someone having to wake up, so it's not likely to put additional load on those people.

I'm not necessarily disagreeing with your overall point, but if you need five nines, you're talking about an entirely different league of infrastructure compared to people who need two or three VMs that could also be handled by a NUC somewhere in the office (which will amortize itself against AWS in a few months).

Re: The cost of cloud

#78
post #25

These types of posts rarely measure operational cost. Anybody can buy infrastructure and stick it behind an API. But can you make it fault tolerant with high availability and low latency. Can you do all of that and _still_ beat AWS's costs? For the vast majority of customers the answer is no. I used to work at a medium sized company, and they saved millions by moving to the cloud, and gained much better availability/…

Ex Amazon here.

I've been in companies that owned their datacenters and it was much, much cheaper than using any cloud service.

Poorly managed datacenters exist but that's an organization problem. Remove the datacenter and you'll have poorly managed cloud instances and services costing millions.

Re: The cost of cloud

#79
post #47

Earlier quoted context omitted.

I don't consider this rude or blunt, but rather incomplete as I really am not sure what points are frustrating for you or what you would hope someone takes away from it -- I'm an outside observer on the cloud subject as I have seen huge debates over use of the public cloud internally at my company and also with client companies. I've seen the billing costs of the public cloud absolutely demolish an IT org's yearly bu…

When I'm frustrated I'm not exactly clear in my writing. You need to be able to do both options before having an opinion on which is appropriate in which case. I am suprised to have to state this. But in my experience people argue one option a lot without being to deliver the other. People who know bare metal are rare these days from the total of available infrastructure engineers (call them sysadmins, devops, etc).…

There are few things we want to do on-premise any more. The main problem of on-premise, and benefit of cloud, is that we can add new capacity at a moments notice. You never have to wonder if you’ll need to add more capacity (with two month lead times) to provision a database.

Now you could say that infra teams that do not anticipate such a need are less than ideal, and I’d agree with you, but I haven’t been part of them and I imagine they have their own issues to deal with.

Cloud (as a dev) makes me not worry about infra teams, since they’re not our problem (beyond the ones managing the cloud environment).

Post reply on HN