Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we use our own hardware

fastmail.com

Post reply on HN
Print view View original

401–410 of 547 posts

Re: Why we use our own hardware

#401

Post by e12e »

jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

I used to help manage a couple of racks worth of on premise hw in early to mid 2000.

We had some old Compaq (?) servers, most of the newer stuff was Dell. Mix of windows and Linux servers.

Even with the Dell boxes, things wasn't really standard across different server generations, and every upgrade was bespoke, except in cases when we bought multiple boxes for redundancy/scaling of a particular service.

What I'd like to see is something like oxide computer servers that scales way down at least down to quarter rack. Like some kind of Supermicro meets backlblaze storage pod - but riffing on Joyent's idea of colocating storage and compute. A sort of composable mainframe for small businesses in the 2020s.

I guess maybe that is part of what Triton is all about.

But anyway - somewhere to start, and grow into the future with sensible redundancies and open source bios/firmware/etc.

Not typical situation for today, where you buy two (for redundancy) "big enough" boxes - and then need to reinvent your setup/deployment when you need two bigger boxes in three years.

Re: Why we use our own hardware

#402

Post by brongondwana »

ksec wrote: post #353
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

>But everyone seemed to be flocking to them. To the point we have young Devs today that dont know what VPS and Colo ( Colocation) meant. Back to the article, I am surprised it was only a "A few years ago" Fastmail adopted SSD. Which certainly seems late in the cycle for the benefits of what SSD offers. Price for Colo on the order of $3000/2U/year. That is $125 /U/month.

We adopted SSD for the current week's email and rust for the deeper storage many years ago. A few years ago we switched to everything on NVMe, so there's no longer two tiers of storage. That's when the pricing switched to make it worthwhile.

Re: Why we use our own hardware

#403

Post by brongondwana »

riezebos wrote: post #365
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

As a customer of Fastmail and a fan of your work at FastAI and FastHTML I feel a bit stupid now for not knowing you started Fastmail. Now I'm wondering how much you'd look like tiangolo if you wore a moustache.

Jeremy is all the Fast things!

Re: Why we use our own hardware

#404

Post by bluGill »

llm_trw wrote: post #373
jph00 wrote: post #344

The original answer to "why does FastMail use their own hardware" is that when I started the company in 1999 there weren't many options. I actually originally used a single bare metal server at Rackspace, which at that time was a small scrappy startup. IIRC it cost $70/month. There weren't really practical VPS or SaaS alternatives back then for what I needed. Rob (the author of the linked article) joined a few months…

>As the fortunes of AWS et al rose and rose and rose, I kept looking at their pricing at features and kept wondering what I was missing. They seemed orders of magnitude more expensive for something that was more complex to manage and would have locked us into a specific vendor's tooling. But everyone seemed to be flocking to them. In 2006 when the first aws instances showed up it would take you two years of on demand…

Aws is useful if you have uneven loads. why pay for the number of servers you need for christmas the rest of the year? But if your load is more even it doesn't make as much sense.

Re: Why we use our own hardware

#405

Post by bluGill »

justsomehnguy wrote: post #399
ksec wrote: post #353

Earlier quoted context omitted.

>But everyone seemed to be flocking to them. To the point we have young Devs today that dont know what VPS and Colo ( Colocation) meant. Back to the article, I am surprised it was only a "A few years ago" Fastmail adopted SSD. Which certainly seems late in the cycle for the benefits of what SSD offers. Price for Colo on the order of $3000/2U/year. That is $125 /U/month.

> Which certainly seems late in the cycle for the benefits of what SSD offers. 90% of emails are never read, 9% are read once. What SSD could offer for this use case except at least 2x cost ?

Don't forget that fastmail is through an internet transport with enough latency to make hdd seek times noise

Re: Why we use our own hardware

#406

Post by e12e »

kwakubiney wrote: post #395

If I remember correctly, StackOverflow does something similar. The then Director of Engineering speaks about it on here[1] [1] https://hanselminutes.com/847/engineering-stack-overflow-wit...

They also have a SaaS product that lives in the cloud:

https://stackoverflow.blog/2023/08/30/journey-to-the-cloud-p...

Re: Why we use our own hardware

#407

Post by jread »

motorest wrote: post #393
jread wrote: post #388

Earlier quoted context omitted.

If have a global user base, depending on your workload, a simple CDN in front of your hardware can often go a long ways with minimal cost and complexity.

> If have a global user base, depending on your workload, a simple CDN in front of your hardware can often go a long ways with minimal cost and complexity. Let's squint hard enough to pretend a CDN does not qualify as "the cloud". That alone requires a lot of goodwill. A CDN distributes read-only content. Any usecase that requires interacting with a service is automatically excluded. So, no.

> Any usecase that requires interacting with a service is automatically excluded

This isn't correct. Many applications consist of a mix of static and dynamic content. Even dynamic content is often cacheable for a time. All of this can be served by a CDN (using TTLs) which is a much simpler and more cost effective solution than multi-region cloud infra, with the same performance benefits.

Re: Why we use our own hardware

#408

Post by brongondwana »

_bare_metal wrote: post #57

Plugging https://BareMetalSavings.com in case you want to ballpark-estimate your move off of the cloud Bonus points: I'm a Fastmail customer, so it tangentially tracks ---- Quick note about the article: ZFS encryption can be flaky, be sure you know what you're doing before deploying for your infrastructure. Relevant Reddit discussion: https://www.reddit.com/r/zfs/comments/1f59zp6/is_zfs_encrypt... A spreadsheet of re…

Yeah, we know about the ZFS encryption with send/receive bug, it's frustrating our attempts to get really nice HA support on our logging system... but so far it appears that just deleting the offsending snapshot and creating a new one works, and we're funding some research into the issue as well.

This is the current script - it runs every minute for each pool synced between the two log servers: https://gist.github.com/brong/6a23fee1480f2d62b8a18ade5aea66...

Re: Why we use our own hardware

#409

Post by RainyDayTmrw »

lukevp wrote: post #351

To me, Cloud is all about the shift left of DevOps. It’s not a cost play. I’m a Dev Lead / Manager and have worked in both types of environments over the last 10 years. It’s immeasurable the velocity difference as far as system provisioning between the two approaches. In the hardware space, it took months to years to provision new machines or upgrade OSes. In the cloud, it’s a new terraform script and a CI deploy awa…

I think I understand your point, and this is not directed at you personally, but: I think "shift left" is another one of those phrases that's lost all meaning, like "synergy" or "agile" before it.

Re: Why we use our own hardware

#410

Post by brongondwana »

bartvk wrote: post #3

Such an awesome article. I like how they didn't just go with the Cloud wave but kept sysadmin'ing, like ol' Unix graybeards. Two interesting things they wrote about their SSDs: 1) "At this rate, we’ll replace these [SSD] drives due to increased drive sizes, or entirely new physical drive formats (such E3.S which appears to finally be gaining traction) long before they get close to their rated write capacity." and 2)…

My beard isn't entirely grey yet!

The new NVMe drives we've only had for a few years, but so far there's only been a single failure across the whole fleet, and we keep spares in stock. It's been very reliable, not like the weeks back in (hmm, 2006? 2007?) the ancient past, when we were losing 15kRPM velociraptors every other day. They had a firmware fault and we eventually got an update which made them reliable, but it was a wild few months.

Post reply on HN