Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we use our own hardware

fastmail.com

Post reply on HN
Print view View original

461–470 of 547 posts

Re: Why we use our own hardware

#461

Post by ForOldHack »

llm_trw wrote: post #373
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

>As the fortunes of AWS et al rose and rose and rose, I kept looking at their pricing at features and kept wondering what I was missing. They seemed orders of magnitude more expensive for something that was more complex to manage and would have locked us into a specific vendor's tooling. But everyone seemed to be flocking to them. In 2006 when the first aws instances showed up it would take you two years of on demand…

"buying the hardware from a retail store." Never buy wholesale and never develop on immature hardware, I have seen c** with multiple 9 y.o. dev servers. I could shorten the ROI to less than 6 months.

Re: Why we use our own hardware

#462

Post by PeterStuer »

llm_trw wrote: post #373
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

>As the fortunes of AWS et al rose and rose and rose, I kept looking at their pricing at features and kept wondering what I was missing. They seemed orders of magnitude more expensive for something that was more complex to manage and would have locked us into a specific vendor's tooling. But everyone seemed to be flocking to them. In 2006 when the first aws instances showed up it would take you two years of on demand…

AWS was built on hordes of VC backed startups drowning in heaps of cash and very little operational expertise.

Re: Why we use our own hardware

#463

Post by BackBlast »

0xbadcafebee wrote: post #446
kapone wrote: post #438

Earlier quoted context omitted.

ok...? But, it comes at a cost. And that cost is significant. Like magnitudes significant. At what point does it become cheaper to hire an infra engineer? Let's see. In the US a good infra engineer might cost you $150K/yr all in. That's not taking into account freelancers/contractors who can do it for less. That's ~$12K/mo. That's a lot of compute on AWS...but that's not the end of the story. Ever try getting data OU…

The bigger cost is what will happen to your business when you're hard-down for a week because all your SQL servers are down, and you don't have spares, and it will take a week to ship new servers and get them racked. Even if you think you could do that very fast, there is no guarantee. I've seen Murphy's Law laugh in the face of assumptions and expectations too many times. But let's not just make vague claims. Everyb…

> The bigger cost is what will happen to your business when you're hard-down for a week because all your SQL servers are down, and you don't have spares, and it will take a week to ship new servers and get them racked. Even if you think you could do that very fast, there is no guarantee. I've seen Murphy's Law laugh in the face of assumptions and expectations too many times.

Lets ignore the loaded, cherry picked situation of no redundancy, no spares, and no warranty service. Because this is all magically hard since cloud providers appeared even though many of us did this, and have done this for years....

There is nothing stopping an on-prem user from renting a replacement from a cloud provider while waiting for hardware to show up. That's a good logical use case for the cloud we can all agree upon.

Next, your cost comparison isn't very accurate. One is isolated dedicated hardware, the other is shared. Junk fees such as egress, IPs, charges for access metal instances, IOPS provisioning for a database, etc will infest the AWS side. The performance of SAN vs local SSD is night and day for a database.

Finally, I can acquire that level of performance hardware much cheaper if I wanted to, order of magnitude is plausible and depends more on where it's located, colo costs, etc.

Re: Why we use our own hardware

#464

Post by jedberg »

tiffanyh wrote: post #457
jedberg wrote: post #451

Earlier quoted context omitted.

In my 25 years, I've run some really big on-prem workloads and some of the biggest cloud loads (Sendmail.org and it's mail servers and Netflix streaming). Here is why I like the cloud: Flexibility. When Netflix wanted to start operating in Europe, we didn't have to negotiate datacenter space, order a bunch of servers, wait for racking and stacking, and all those other things. We just made an API call and had an entir…

When you say “cloud”, are you including old school web hosts that will rent you a dedicated server? Like OVH, Hetzner or Hivelocity? Because you can get some insane servers for like $300/month (eg brand new 5th gen Epyc 48-core / 0.5TB ram / lots of NVME) and globally available.

Those could count. But you'll still end up having to do some linux admin, which a lot of people can't do anymore.

The whole point is that the closer you can get to "write code, run code", the faster you can launch and innovate.

Re: Why we use our own hardware

#465

Post by lowsong »

jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

What is the software side of things like? Is your team managing these servers directly — or is it "cloud like" with containers (Kubernetes?), IaC tools, etc.

Re: Why we use our own hardware

#466

Post by favflam »

johnklos wrote: post #65

The whole push to the cloud has always fascinated me. I get it - most people aren't interested in babysitting their own hardware. On the other hand, a business of just about any size that has any reasonable amount of hosting is better off with their own systems when it comes purely to cost. All the pro-cloud talking points are just that - talking points that don't persuade anyone with any real technical understanding…

I did compliance for a fintech under heavy regulation.

If we used AWS, we could skip months of certification. If we use a custom data center, we have to certify it ourselves (muuuuuch more expensive).

From this standpoint, cloud beats on-premise.

Re: Why we use our own hardware

#467

Post by jb1991 »

christophilus wrote: post #30
jb1991 wrote: post #26

Earlier quoted context omitted.

That probably says more about the email address that’s out there than anything else.

Fastmail has wildcard email support, so it’s pretty easy to have an email per purchase you make (for example). This makes it easy to see who leaked your email to spammers. Anyway, I have nowhere near the volume of spam with Fastmail that I had with Gmail.

My point was not about wildcard emails, which Gmail also offers. Rather, the amount of spam you get is typically based on how well known your email address is to spammers. If someone’s not getting much spam, it usually just means they haven’t used their email address in places where they would get it. This is regardless of whether it’s a wildcard email or not.

Re: Why we use our own hardware

#468

Post by fulafel »

nine_k wrote: post #339
fulafel wrote: post #194

Earlier quoted context omitted.

OTOH: 1. big clouds are very lucrative targets for spooks, your data seem pretty likely to be hoovered up as "bycatch" (or maybe main catch depending on your luck) by various agencies and then traded around as currency 2. you never hear about security probems (incidents or exposure) in the platforms, there's no transparency 3. better than most coporate stuff is a low bar

If you don't want your data to be accessible to "various agencies", don't share it with corporations, full stop. Corporations are obliged by law to make it available to the agencies, and the agencies often overreach, while the corporations almost never mind the overreach. There are limitations for stuff like health or financial data, but these are not impenetrable barriers. I would just consider all your hosted data…

There's much variation by jurisdiction. Eg US based big-cloud companies would seem more risky here if you're from a country with traditionally less invasive (and less funded) spooks.

Re: Why we use our own hardware

#469

Post by mbrumlow »

benterix wrote: post #358
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

> As the fortunes of AWS et al rose and rose and rose, I kept looking at their pricing at features and kept wondering what I was missing. You are not the only one. There are several factors at play but I believe one of the strongest today is the generational divide: the people lost the ability to manage their own infra or don't know it well enough to do it well so it's true when they say "It's too much hassle". I say…

> I don't believe the lack of skills is the main reason organizations have problem

IDK. More and more I see the argument of “I don’t know, and we are not experts in xxx” as a winning argument of why we should just spend money on 3rd party services and products.

I have seen people getting paid 700k plus a year spend their entire stay at companies writing papers about how they can’t do something and the obvious solution is to spend 400k plus to have some 3rd party handle it, and getting the budget.

Let’s not get into what the conversation looks like when somebody points out that we might have an issue if we are paying somebody 700k to hire somebody else temporarily for 400k each year, and that we should find these folks who can do it for 400k and just hire Them.

All this to say that being a SWE in many companies today requires no ability to create software that solves business problems. But rather some sort of quasi system administrator manager who will maybe write a handful of DSL scripts over the course of their career.

Re: Why we use our own hardware

#470

Post by prmoustache »

leshenka wrote: post #442
bluGill wrote: post #404

Earlier quoted context omitted.

Aws is useful if you have uneven loads. why pay for the number of servers you need for christmas the rest of the year? But if your load is more even it doesn't make as much sense.

But how does it look from aws point of view? Everyone scales up around Christmas then scales down afterwards. What do THEY do with all the unneeded CPU-seconds for the rest of the year?

We certainly don't scale up around Christmas. Appart from online shops and shipping companies, why would everyone else scale up around Christmas?
Post reply on HN