Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we use our own hardware

fastmail.com

Post reply on HN
Print view View original

151–160 of 547 posts

Re: Why we use our own hardware

#151

Post by UltraSane »

johnklos wrote: post #65

The whole push to the cloud has always fascinated me. I get it - most people aren't interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost. All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding…

"All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding,"

This is false. AWS infrastructure is vastly more secure than almost all company data centers. AWS has a rule that the same person cannot have logical access and physical access to the same storage device. Very few companies have enough IT people to have this rule. The AWS KMS is vastly more secure than what almost all companies are doing. The AWS network is vastly better designed and operated than almost all corporate networks. AWS S3 is more reliable and scalable than anything almost any company could create on their own. To create something even close to it you would need to implement something like MinIO using 3 separate data centers.

Re: Why we use our own hardware

#152

Post by AnthonyMouse »

tylerchurch wrote: post #123
sgarland wrote: post #103

Earlier quoted context omitted.

I’m not convinced this is entirely true. The upfront cost if you don’t have the skills, sure – it takes time to learn Linux administration, not to mention management tooling like Ansible, Puppet, etc. But once those are set up, how is it different? AWS is quite clear with their responsibility model that you still have to tune your DB, for example. And for the setup, just as there are Terraform modules to do everythin…

I think this is only true for teams and apps of a certain size. I've worked on plenty of teams with relatively small apps, and the difference between: 1. Cloud: "open up the cloud console and start a VM" 2. Owned hardware: "price out a server, order it, find a suitable datacenter, sign a contract, get it racked, etc." Is quite large. #1 is 15 minutes for a single team lead. #2 requires the team to agree on hardware s…

You're assuming that hosting something in-house implies that each application gets its own physical server.

You buy a couple of beastly things with dozens of cores. You can buy twice as much capacity as you actually use and still be well under the cost of cloud VMs. Then it's still VMs and adding one is just as fast. When the load gets above 80% someone goes through the running VMs and decides if it's time to do some house cleaning or it's time to buy another host, but no one is ever waiting on approval because you can use the reserve capacity immediately while sorting it out.

Re: Why we use our own hardware

#153

Post by sgarland »

floating-io wrote: post #114
jeffbee wrote: post #75

Earlier quoted context omitted.

The problem with your claims here is they can only be right if the entire industry is experiencing mass psychosis. I reject a theory that requires that, because my ego just isn't that large. I once worked for several years at a publicly traded firm well-known for their return-to-on-prem stance, and honestly it was a complete disaster. The first-party hardware designs didn't work right because they didn't have the har…

"Vendor problems" is a red herring, IMO; you can have those in the cloud, too. It's been my experience that those who can build good, reliable, high-quality systems, can do so either in the cloud or on-prem, generally with equal ability. It's just another platform to such people, and they will use it appropriately and as needed. Those who can only make it work in the cloud are either building very simple systems (whi…

> or are building a house of cards that will eventually collapse (or just cost them obscene amounts of money to keep on life support)

Ding ding ding. It's this.

> The whole CapEx vs. OpEx issue drives me batty

Seconded. I can't help but feel like it's not just a "I don't understand money" thing, but more of a "the way Wall Street assigns value is fundamentally broken." Spending $100K now, once, vs. spending $25K/month indefinitely does not take a genius to figure out.

Re: Why we use our own hardware

#154

Post by necovek »

tylerchurch wrote: post #123
sgarland wrote: post #103

Earlier quoted context omitted.

I’m not convinced this is entirely true. The upfront cost if you don’t have the skills, sure – it takes time to learn Linux administration, not to mention management tooling like Ansible, Puppet, etc. But once those are set up, how is it different? AWS is quite clear with their responsibility model that you still have to tune your DB, for example. And for the setup, just as there are Terraform modules to do everythin…

I think this is only true for teams and apps of a certain size. I've worked on plenty of teams with relatively small apps, and the difference between: 1. Cloud: "open up the cloud console and start a VM" 2. Owned hardware: "price out a server, order it, find a suitable datacenter, sign a contract, get it racked, etc." Is quite large. #1 is 15 minutes for a single team lead. #2 requires the team to agree on hardware s…

Before the cloud, you could get a VM provisioned (virtual servers) or a couple of apps set up (LAMP stack on a shared host ;)) in a few minutes over a web interface already.

"Cloud" has changed that by providing an API to do this, thus enabling IaC approach to building combined hardware and software architectures.

Re: Why we use our own hardware

#155

Post by UltraSane »

icedchai wrote: post #132
cpursley wrote: post #73

Earlier quoted context omitted.

The fact is, managing your own hardware is a pita and a distraction from focusing on the core product. I loathe messing with servers and even opt for "overpriced" paas like fly, render, vercel. Because every minute messing with and monitoring servers is time not spent on product. My tune might change past a certain size and a massive cloud bill and there's room for full time ops people, but to offset their salary, it…

Writing piles of IaC code like Terraform and CloudFormation is also a PITA and a distraction from focusing on your core product. PaaS is probably the way to go for small apps.

But that effort has a huge payoff in that it can be used to disaster recovery in a new region and to spin up testing environments.

Re: Why we use our own hardware

#156

Post by sgarland »

oftenwrong wrote: post #80
wongarsu wrote: post #37

Earlier quoted context omitted.

For so many things the cloud isn't really easier or cheaper, and most cloud providers stopped advertising it as such. My assumption is that cloud adoption is mainly driven by 3 forces: - for small companies: free credits - for large companies: moving prices as far away as possible from the deploy button, allowing dev and it to just deploy stuff without purchase orders - self-perpetuating due to hype, cv-driven develo…

In small companies, cloud also provides the ability to work around technical debt and to reduce risk. For example, I have seen several cases where poorly designed systems that unexpectedly used too much memory, and there was no time to fix it, so the company increased the memory on all instances with a few clicks. When you need to do this immediately to avoid a botched release that has already been called "successful…

> For example, I have seen several cases where poorly designed systems that unexpectedly used too much memory

> using a managed database system like RDS vs self-managing the same RDBMS: using the managed version saves on labor

As a DBRE / SRE, I can confidently assert that belief in the latter is often directly responsible for the former. AWS is quite clear in their shared responsibility model [0] that you are still responsible for making sound decisions, tuning various configurations, etc. Having staff that knows how to do these things often prevents the poor decisions from being made in the first place.

[0]: https://aws.amazon.com/compliance/shared-responsibility-mode...

Re: Why we use our own hardware

#157

Post by noprocrasted »

jeffbee wrote: post #99
noprocrasted wrote: post #85

Earlier quoted context omitted.

That argument makes sense for PaaS services like the ones you mention. But for bare "cloud" like AWS, I'm not convinced it is saving any effort, it's merely swapping one kind of complexity with another. Every place I've been in had full-time people messing with YAML files or doing "something" with the infrastructure - generally trying to work around the (self-inflicted) problems introduced by their cloud provider - w…

In what sense is AWS "bare cloud"? S3, DynamoDB, Lambda, ECS?

How do you configure S3 access control? You need to learn & understand how their IAM works.

How do you even point a pretty URL to a lambda? Last time I looked you need to stick an "API gateway" in front (which I'm sure you also get nickel & dimed for).

How do you go from "here's my git repo, deploy this on Fargate" with AWS? You need a CI pipeline which will run a bunch of awscli commands.

And I'm not even talking about VPCs, security groups, etc.

Somewhat different skillsets than old-school sysadmin (although once you know sysadmin basics, you realize a lot of these are just the same concepts under a branded name and arbitrary nickel & diming sprinkled on top), but equivalent in complexity.

Re: Why we use our own hardware

#158

Post by indulona »

I am working on a personal project(some would call it startup, but i have no intention of getting external financing and other americanisms) where i have set up my own cdn and video encoding, among other things. These days, whenever you have a problem, everyone answers "just use cloud" and that results in people really knowing nothing any more. It is saddening. But on the other hand it ensures all my decades of knowledge will be very well paid in the future, if i'd need to get a job.

Re: Why we use our own hardware

#159

Post by sroussey »

cpursley wrote: post #79
cpursley wrote: post #73

Earlier quoted context omitted.

The fact is, managing your own hardware is a pita and a distraction from focusing on the core product. I loathe messing with servers and even opt for "overpriced" paas like fly, render, vercel. Because every minute messing with and monitoring servers is time not spent on product. My tune might change past a certain size and a massive cloud bill and there's room for full time ops people, but to offset their salary, it…

Anecdotal - but I once worked for a company where the product line I built for them after acquisition was delayed by 5 months because that's how long it took to get the hardware ordered and installed in the datacenter. Getting it up on AWS would have been a days work, maybe two.

Oh, absolutely, with your own hardware you need planning. Time to deployment is definitely a thing.

Really, the one major thing that bites on cloud providers in there 99.9% margin on egress. The markup is insane.

Re: Why we use our own hardware

#160

Post by daemonologist »

zosima wrote: post #83
johnklos wrote: post #65

The whole push to the cloud has always fascinated me. I get it - most people aren't interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost. All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding…

Cloud expands the capabilities of what one team can manage by themselves, enabling them to avoid a huge amount of internal politics. This is worth astronomical amounts of money in big corps.

Our big company locked all cloud resources behind a floating/company-wide DevOps team (git and CI too). We have an old on-prem server that we jealously guard because it allows us to create remotes for new git repos and deploy prototypes without consulting anyone.

(To be fair, I can see why they did it - a lot of deployments were an absolute mess before.)

Post reply on HN