Live data from Hacker News

Rolling your own servers with Kubernetes

gravitational.com

111–120 of 209 posts

Re: Rolling your own servers with Kubernetes

#111
post #53

>Should you roll your own servers? If you are not certain, the answer is most likely “no”. The staggering growth of AWS happened for a reason. Funny how for many decades companies and people were running their own servers. The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to…

Yep and we were running bare metal servers with pretty bad behaviour sec wise.

Re: Rolling your own servers with Kubernetes

#112

Earlier quoted context omitted.

The hardware isn’t the expensive part, it’s the people to run it that is.

Using publicly available costing data from last November, I see that an m4.xlarge RDS reserved instance of SQL Server Enterprise Edition with multi-AZ failover is $39,000 / year. If I have 4 DBAs at a cost of $150K / yr salary + benefits, that's the equivalent of around 15 smallish, managed SQL Server instances. My current SQL footprint is on the order of 600 instances, which works out to be a bit over $23M a year. I…

As your operation gets bigger, the scale generally starts to tip in favor of running your own hardware. But 600 SQL Server instances is fairly large—when that number is in the single digits, the cost of DBAs/DevOps dwarfs that of the hardware. So the scale of your business matters a lot, and it's not useful to generalize this to all companies.

Edit: Your math seems off as well. Don't forget that you'd have to pay the SQL Server license regardless of where it's hosted, as well as the cost of dedicated hardware if you go that route, so the difference between the two options will be much smaller than $39k per instance. And I don't know if that salary number is real, but $150k including overhead seems really low for a good DBA (at least in the US)—I would expect at least $200k.

Re: Rolling your own servers with Kubernetes

#113
post #28

Earlier quoted context omitted.

Counterpoint: These issues still exist, you just lose visibility into them by putting someone else in between. And usually, the someone else is big enough that you're not a priority.

with limited resource, you have to choose to balance between the stability or more control over your infra. IMO, small companies can't afford both, but also likely they would not need both at that size.

B.S. I was solo IT for a 50+ppl company, on a three-person team with a CIO/DBA and a data manager. I supported a full Windows/Exchange LAN with heavy printing needs, tape backups, internal/external infrastructural services and webservers with LOB java webapps, remote users, etc. Through all this I learned Perl from scratch, rearchitected the business systems (hardware) and reduced their production time by like 90% (two weeks to two days) with zero downtime (primordial blue-green, as it turns out), and wrote a completely new ETL flow with my new Perl skills. From four servers under a table to four racks, SaaS architecture is exceedingly simple next to all this, and the company spent probably around $10K/mo all told, maybe up to $20k while I was buying hardware.

Maybe you simply haven't seen this kind of operation in action before?

Re: Rolling your own servers with Kubernetes

#114

Earlier quoted context omitted.

Using publicly available costing data from last November, I see that an m4.xlarge RDS reserved instance of SQL Server Enterprise Edition with multi-AZ failover is $39,000 / year. If I have 4 DBAs at a cost of $150K / yr salary + benefits, that's the equivalent of around 15 smallish, managed SQL Server instances. My current SQL footprint is on the order of 600 instances, which works out to be a bit over $23M a year. I…

As your operation gets bigger, the scale generally starts to tip in favor of running your own hardware. But 600 SQL Server instances is fairly large—when that number is in the single digits, the cost of DBAs/DevOps dwarfs that of the hardware. So the scale of your business matters a lot, and it's not useful to generalize this to all companies. Edit: Your math seems off as well. Don't forget that you'd have to pay the…

Well of course it wasn't strictly apples-to-apples because I did leave off the cost of the DCs, the hardware, and the initial license acquisition. But those costs are shared across multiple teams, amortized over long lifespans, etc, and we also get volume discounts. But even taking that all into account - for 24M we can, and have, bought about a PB of storage, multiple blades and enclosures, licensed OS, and acommodate the salaries of the teams we need to run it all. I'd hazard a guess that that 24M is a pretty significant portion of the entire IT budget, in fact.

Re: Rolling your own servers with Kubernetes

#115
post #60

Earlier quoted context omitted.

It usually takes the form of "ugh then you have to hire dedicated people to manage it."

Which is a valid argument, isn't it? Engineers are expensive! Whether they're more expensive than AWS depends entirely on what you're doing, but the answer is almost always "yes" at a small enough scale.

See my other comment.

Re: Rolling your own servers with Kubernetes

#116
post #53

>Should you roll your own servers? If you are not certain, the answer is most likely “no”. The staggering growth of AWS happened for a reason. Funny how for many decades companies and people were running their own servers. The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to…

That's the difference between comparative advantage and absolute advantage.

Re: Rolling your own servers with Kubernetes

#117

Earlier quoted context omitted.

Using publicly available costing data from last November, I see that an m4.xlarge RDS reserved instance of SQL Server Enterprise Edition with multi-AZ failover is $39,000 / year. If I have 4 DBAs at a cost of $150K / yr salary + benefits, that's the equivalent of around 15 smallish, managed SQL Server instances. My current SQL footprint is on the order of 600 instances, which works out to be a bit over $23M a year. I…

How much is the license for SQL enterprise nowadays? still 5k per node or does Microsoft charge per socket now?

Depends on your organization, the licensing model you pick, etc. They generally license per core now, and you have to run at least 4 cores. Without giving too much away I'd say the cost of licensing a 4-core box running Enterprise is on the order of $20K, which you pay one time, up front. Then you just pay for your support contract. You can also do stuff like license an entire VMware farm or failover cluster, and then you can overprovision services for extra cost savings.

Re: Rolling your own servers with Kubernetes

#118

Earlier quoted context omitted.

The hardware isn’t the expensive part, it’s the people to run it that is.

Using publicly available costing data from last November, I see that an m4.xlarge RDS reserved instance of SQL Server Enterprise Edition with multi-AZ failover is $39,000 / year. If I have 4 DBAs at a cost of $150K / yr salary + benefits, that's the equivalent of around 15 smallish, managed SQL Server instances. My current SQL footprint is on the order of 600 instances, which works out to be a bit over $23M a year. I…

Do you have multi-AZ failover now? And probably many of your 600 instances would be fine on a lower tier? On Azure (as an example) one gets 100 of the lowest managed SQL DB tier per 150K/year developer.

Re: Rolling your own servers with Kubernetes

#119

Earlier quoted context omitted.

Using publicly available costing data from last November, I see that an m4.xlarge RDS reserved instance of SQL Server Enterprise Edition with multi-AZ failover is $39,000 / year. If I have 4 DBAs at a cost of $150K / yr salary + benefits, that's the equivalent of around 15 smallish, managed SQL Server instances. My current SQL footprint is on the order of 600 instances, which works out to be a bit over $23M a year. I…

As your operation gets bigger, the scale generally starts to tip in favor of running your own hardware. But 600 SQL Server instances is fairly large—when that number is in the single digits, the cost of DBAs/DevOps dwarfs that of the hardware. So the scale of your business matters a lot, and it's not useful to generalize this to all companies. Edit: Your math seems off as well. Don't forget that you'd have to pay the…

I should also have added - SQL on RDS, the license cost is baked in. You have to re-purchase the license every year. You don't have to do that with enterprise licensing, which is a one-time upfront cost. You also can't overprovision, and you can't use alternate licensing models like MSDN.

Re: Rolling your own servers with Kubernetes

#120

Earlier quoted context omitted.

Using publicly available costing data from last November, I see that an m4.xlarge RDS reserved instance of SQL Server Enterprise Edition with multi-AZ failover is $39,000 / year. If I have 4 DBAs at a cost of $150K / yr salary + benefits, that's the equivalent of around 15 smallish, managed SQL Server instances. My current SQL footprint is on the order of 600 instances, which works out to be a bit over $23M a year. I…

Do you have multi-AZ failover now? And probably many of your 600 instances would be fine on a lower tier? On Azure (as an example) one gets 100 of the lowest managed SQL DB tier per 150K/year developer.

On tier 1 stuff, yeah. We certainly have a number of instances that would be fine on a lower tier, like dev instances etc that we can license under MSDN subscriptions, or that honestly would be better off as containers on individual devs' machines. But AWS doesn't offer that, and from those same numbers I see that a single-AZ instance is still $19K a year, every year. That's actually new cost because we can't BYOL for RDS.
Post reply on HN