Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we use our own hardware

fastmail.com

Post reply on HN
Print view View original

431–440 of 547 posts

Re: Why we use our own hardware

#431

Post by jph00 »

0xbadcafebee wrote: post #375
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

You know how to set up a rock-solid remote hands console to all your servers, I take it? Dial-up modem to a serial console server, serial cables to all the servers (or IPMI on a segregated network and management ports). Then you deal with varying hardware implementations, OSes, setting that up in all your racks in all your colos. Compare that to AWS, where there are 6 different kinds of remote hands, that work on all…

It's really not like that at all. If it was, I expect after 25 years of growth FastMail would probably have noticed. Much of what you're describing assumes a poorly run company that isn't able to make good choices -- if you have such a mix of odd hardware os OSes then that's pretty bad sign.

Prioritise simplicity.

For remote hands, 2 kinds is sufficient: IP KVM, and an actual person walking over to your machine. Can't say I've had an AWS person talk to me on a cell phone whilst standing at my server to help me sort out an issue.

It's actually really fun, and saving 90% what can be your largest cost can actually be a fundamental driver of startup success. You can undercut the competition on price and offer stuff that's just not available otherwise.

Every time this conversation has come up online over the last few decades there's always a few people who parrot this claim it's all too hard. I can't imagine these comments come from people that have actually gone and done it.

Re: Why we use our own hardware

#432

Post by jph00 »

packtreefly wrote: post #356
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

> although I was worried that folks are too locked in to SaaS stuff For some people the cloud is straight magic, but for many of us, it just represents work we don't have to do. Let "the cloud" manage the hardware and you can deliver a SaaS product with all the nines you could ask for... > teaching a course on how to do all this ... there might be interest in that after all? Idk about a course, but I'd be interested…

You're describing stuff the colo provider does. I have no plans to describe how to setup a colo provider. I've never done that, and haven't seen the need. The cost of colo is not that significant.

Re: Why we use our own hardware

#433

Post by Beijinger »

mariusor wrote: post #273
Beijinger wrote: post #207

I was told Fastmail is excellent, and I am not a big fan of gmail. Once locked out for good in gmail, your email and apps associated with it, are gone forever. Source? Personal experience. "A private inbox $60 for 12 months". I assume it is USD, not AU$ (AFAIK, Fastmail is based in Australia.) Still pricey. At https://www.infomaniak.com/ I can buy email service for an (in my case external) domain for 18 Euro a year a…

My suggestion would be to try Purelymail. They don't offer much in the way of a web interface to email, but if you bring your own client, it's a very good provider. I'm paying something like $10 per year for multiple domains with multiple email addresses (though with little traffic). I've been using them for about 5 years and I had absolutely no issues.

Pricing it hard to understand: https://purelymail.com/advancedpricing

Usernames on shared domains:

1 to 6 letters: $0.20 per user per year 7 to 12 letters: $0.05 per user per year 13+ letters: $0.02 per user per year

WTF?

Re: Why we use our own hardware

#434

Post by UltraSane »

stefan_ wrote: post #297
fulafel wrote: post #194

Earlier quoted context omitted.

OTOH: 1. big clouds are very lucrative targets for spooks, your data seem pretty likely to be hoovered up as "bycatch" (or maybe main catch depending on your luck) by various agencies and then traded around as currency 2. you never hear about security probems (incidents or exposure) in the platforms, there's no transparency 3. better than most coporate stuff is a low bar

4. we keep hitting hypervisor bugs and having to work around the fact that your software coexists on the same machine with 3rdparty untrusted software who might in fact be actively trying to attack you. All this silliness with encrypted memory buses and the various debilitating workarounds for silicon bugs. So yes, the cloud is very secure, except for the very thing that makes it the cloud that is not secure at all a…

What hypervisor bugs are you referring to? AWS does offer bare metal servers.

Re: Why we use our own hardware

#435

Post by UltraSane »

AtlasBarfed wrote: post #311
UltraSane wrote: post #235

Earlier quoted context omitted.

Encrypting data is easy, securely managing keys is the hard part. KMS is the Key Management Service. And AWS put a lot of thought and work into it. https://docs.aws.amazon.com/kms/latest/cryptographic-details...

It's now been two years since I used KMS, but at the time it seemed little more than S3 API interface with Twitter size limitations Fundamentally why would KMS be more secure than S3 anyway? Both ultimately have the same fundamental security requirements and do the same thing. So the big whirlydoo is KMS has hardware keygen. im sorry, that sounds like something almost guaranteed to have nsa backdoor, or has so much n…

If the NSA is part of your threat model then good luck. I'm not sure any single company could withstand the NSA really trying to hack them for years. The threat of possible NSA backdoors is not a reasonable argument against a cloud provider as the NSA could also have backdoors in every CPU AMD and Intel and AWS makes.

Re: Why we use our own hardware

#436

Post by UltraSane »

likeabatterycar wrote: post #338
UltraSane wrote: post #151

Earlier quoted context omitted.

"All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding," This is false. AWS infrastructure is vastly more secure than almost all company data centers. AWS has a rule that the same person cannot have logical access and physical access to the same storage device. Very few companies have enough IT people to have this rule. The AWS KMS is vastly m…

AWS hires the same cretins that inhabit every other IT department, they just usually happen to be more technically capable. That doesn't make them any more or less trustworthy or reliable.

"cretins"?

Re: Why we use our own hardware

#437

Post by UltraSane »

wslh wrote: post #316
UltraSane wrote: post #259

Earlier quoted context omitted.

Cloud providers are very big targets but have enormous economic incentive to be secure and thus have very large teams of very competent security experts.

You can have full security competence but be a rogue actor at the same time.

I don't understand what this is trying to say.

Re: Why we use our own hardware

#438

Post by kapone »

0xbadcafebee wrote: post #375
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

You know how to set up a rock-solid remote hands console to all your servers, I take it? Dial-up modem to a serial console server, serial cables to all the servers (or IPMI on a segregated network and management ports). Then you deal with varying hardware implementations, OSes, setting that up in all your racks in all your colos. Compare that to AWS, where there are 6 different kinds of remote hands, that work on all…

ok...?

But, it comes at a cost. And that cost is significant. Like magnitudes significant.

At what point does it become cheaper to hire an infra engineer? Let's see.

In the US a good infra engineer might cost you $150K/yr all in. That's not taking into account freelancers/contractors who can do it for less.

That's ~$12K/mo.

That's a lot of compute on AWS...but that's not the end of the story. Ever try getting data OUT of AWS? Yeah, those egress costs are not chump change. But that's not even the end of it.

The more important question is, what's the ratio of hosting/cloud costs to overall revenue? If colo/owned DC will yield better financials over ~few quarters, you'd be bananas as a CTO to recommend the cloud.

Re: Why we use our own hardware

#439

Post by champtar »

UltraSane wrote: post #235
evantbyrne wrote: post #223

Earlier quoted context omitted.

Making API calls from a VM on shared hardware to KMS is vastly more secure than doing AES locally? I'm skeptical to say the least.

Encrypting data is easy, securely managing keys is the hard part. KMS is the Key Management Service. And AWS put a lot of thought and work into it. https://docs.aws.amazon.com/kms/latest/cryptographic-details...

You can securely store your asymmetric key for signing, but if I remember correctly the logs are pretty useless, basically you just know the key was used to make a signature, no option to log the signature or additional metadata, which would help auditing after an account/app compromise.

Re: Why we use our own hardware

#440

Post by kapone »

ksec wrote: post #353
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

>But everyone seemed to be flocking to them. To the point we have young Devs today that dont know what VPS and Colo ( Colocation) meant. Back to the article, I am surprised it was only a "A few years ago" Fastmail adopted SSD. Which certainly seems late in the cycle for the benefits of what SSD offers. Price for Colo on the order of $3000/2U/year. That is $125 /U/month.

What??

I can get an entire rack at Equinix for ~1200/mo with an unlimited 10g internet connect.

Post reply on HN