Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we use our own hardware

fastmail.com

Post reply on HN
Print view View original

321–330 of 547 posts

Re: Why we use our own hardware

#321

Post by noprocrasted »

motorest wrote: post #253
johnklos wrote: post #65

The whole push to the cloud has always fascinated me. I get it - most people aren't interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost. All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding…

> All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding,(...) This is where you lose all credibility. I'm going to focus on a single aspect: performance. If you're serving a global user base and your business, like practically all online businesses, is greatly impacted by performance problems, the only solution to a physics problem is to deplo…

The complexity of scaling out an application to be closer to the users has never been about getting the hardware closer. It's always about how do you get the data there and dealing with the CAP theorem, which requires hard tradeoffs to be decided on when designing the application and can't be just tacked on - there is no magic button to do this, in the AWS console or otherwise.

Getting the hardware closer to the users has always been trivial - call up any of the many hosting providers out there and get a dedicated server, or a colo and ship them some hardware (directly from the vendor if needed).

Re: Why we use our own hardware

#322

Post by fnord77 »

johnklos wrote: post #65

The whole push to the cloud has always fascinated me. I get it - most people aren't interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost. All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding…

capex vs opex

Re: Why we use our own hardware

#323

Post by maccard »

cyberax wrote: post #303
maccard wrote: post #175

Earlier quoted context omitted.

I work for a 50 person subsidiary of a 30k person organisation. I needed a domain name. I put in the purchase request and 6 months later eventually gave up, bought it myself and expensed it. Our AWS account is managed by an SRE team. It’s a 3 day turnaround process to get any resources provisioned, and if you don’t get the exact spec right (you forgot to specify the iops on the volume? Oops) 3 day turnaround. Already…

> Our AWS account is managed by an SRE team. That's an anti-pattern (we call it "the account") in the AWS architecture. AWS internally just uses multiple accounts, so a team can get their own account with centrally-enforced guardrails. It also greatly simplifies billing.

That’s not something that I have control over or influence over.

Re: Why we use our own hardware

#324

Post by kayson »

Any ideas how they manage the ZFS encryption key? I've always wondered what you'd do in an enterprise production setting. Typing the password in at a prompt as any seem scalable (but maybe they have few enough servers that it's manageable) and keeping it in a file on disk or on removable storage would seem to defeat the purpose...

Re: Why we use our own hardware

#325

Post by maccard »

mbesto wrote: post #318
maccard wrote: post #175

Earlier quoted context omitted.

I work for a 50 person subsidiary of a 30k person organisation. I needed a domain name. I put in the purchase request and 6 months later eventually gave up, bought it myself and expensed it. Our AWS account is managed by an SRE team. It’s a 3 day turnaround process to get any resources provisioned, and if you don’t get the exact spec right (you forgot to specify the iops on the volume? Oops) 3 day turnaround. Already…

> 3 day turnaround process to get any resources provisioned Now imagine having to deal with procurement to purchase hardware for your needs. 6 months later you have a server. Oh you need a SAN for object storage? There goes another 6 months.

At a previous job we had some decent on prem resources for internal services. The SRE guys had a bunch of extra compute and you would put in a ticket for a certain amount of resources (2 cpu, SSD, 8GB memory x2 on different hosts). There wasn’t a massive amount of variability between the hardware, and you just requested resources to be allocated from a bunch of hypervisors. Turnaround time was about 3 days too. Except, you were t required to be self sufficient in AWS terminology to request exactly what you needed .

Re: Why we use our own hardware

#326

Post by klysm »

herf wrote: post #279

zfs encryption is still corrupting datasets when using zfs send/receive for backup (huge win for mail datasets), would be cautious about using it in production: https://github.com/openzfs/zfs/issues/12014

I’ll never use ZFS in production after I was on a team that used it at petabyte scale. It’s too complex and tries to solve problems that should be solved at higher layers.

Re: Why we use our own hardware

#327

Post by [deleted] »

sgarland wrote: post #90
johnklos wrote: post #65

The whole push to the cloud has always fascinated me. I get it - most people aren't interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost. All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding…

> What's particularly fascinating to me, though, is how some people are so pro-cloud that they'd argue with a writeup like this with silly cloud talking points. I’m sure I’ll be downvoted to hell for this, but I’m convinced that it’s largely their insecurities being projected. Running your own hardware isn’t tremendously difficult, as anyone who’s done it can attest, but it does require a much deeper understanding of…

[deleted]

Re: Why we use our own hardware

#328

Post by dietr1ch »

bobnamob wrote: post #310
graemep wrote: post #98

Earlier quoted context omitted.

Management in many places will not even know what multi-cloud is (or even multi-region). As Cloudstrike showed, if you follow the crowd and tick the right boxes you will not be blamed.

nit: Crowdstrike Unless the incident is now being referred to as “Cloudstrike”, in which case, eww

Yeah, he meant Crowdstrike. Cloudstrike is the name of a future security incident affecting multiple cloud provides. I can't disclose more details.

Re: Why we use our own hardware

#329

Post by ttul »

johnklos wrote: post #65

The whole push to the cloud has always fascinated me. I get it - most people aren't interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost. All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding…

My firm belief after building a service at scale (tens of millions of end users, > 100K tps) is that AWS is unbeatable. We don’t even think about building our own infrastructure. There’s no way we could ever make it reliable enough, secure enough, and future-proof enough to ever pay back the cost difference.

Something people neglect to mention when they tout their home grown cloud is that AWS spends significant cycles constantly eliminating technical debt that would absolutely destroy most companies - even ones with billion dollar services of their own. The things you rely on are constantly evolving and changing. It’s hard enough to keep up at the high level of a SaaS built on top of someone else’s bulletproof cloud. But imagine also having to keep up with the low level stuff like networking and storage tech?

No thanks.

Re: Why we use our own hardware

#330

Post by johnklos »

sanderjd wrote: post #174
johnklos wrote: post #65

The whole push to the cloud has always fascinated me. I get it - most people aren't interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost. All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding…

Also, by the way, I found it interesting that you framed your side of this disagreement as the technically correct one, but then included this: > a desire to not centralize the Internet This is an ideological stance! I happen to share this desire. But you should be aware of your own non-technical - "emotional" - biases when dismissing the arguments of others on the grounds that they are "emotional" and+l "fanatical".

I never said that my own reasons were neither personal nor emotional. I was just pointing out that my reasons are easy to articulate.

I do think it's more than just emotional, though, but most people, even technical people, haven't taken the time to truly consider the problems that will likely come with centralization. That's a whole separate discussion, though.

Post reply on HN