Live data from Hacker News

11 years of hosting a SaaS

ghiculescu.substack.com

201–210 of 222 posts

Re: 11 years of hosting a SaaS

#201

Would be interested to know what your hosting bill was like? We recently moved everything in-house and started hosting our own servers. Having physical access to the machine (and the network HW) pretty much eliminated all of our dev ops (1). We expected the HW to fail often and such, but in 2 years the only maintenance we had to do was add more disks to the RAID array and resize the partition a few times. Even though…

Does geolocation of the server not matter for your application? I am curious if there are any latency issues with this solution as opposed to a hosted option.

Re: 11 years of hosting a SaaS

#202
post #17

Earlier quoted context omitted.

It may be a generational thing, a matter of familiarity with computing and computers. For someone who's lived through the 80s, "handrolling postgres" doesn't sound nearly as scary as you imagine. I expect the cost/benefit analysis of "handrolling postgres on a beefy Hetzner server" vs "navigating the menus and options of AWS services" would be different for different teams.

I'm looking at the ansible playbooks to setup my favourite beefy baremetal Hetzner server (128GB ram, Ryzen 9 5950X 16-Core, 450Gb fast NVME SSD, 3.5TB x2 NVME SSDs, 155€/month): - Install Debian 11 while booted in rescue mode. - Setup the root file system encryption using cryptsetup and dropbear (to enter the key during the boot through SSH). Involves chroot and some fun commands. - Setup ZFS encrypted mirror filesy…

and you absolutely can't be sure your saas vendor is doing encryption and hardening.

Re: 11 years of hosting a SaaS

#203

Would be interested to know what your hosting bill was like? We recently moved everything in-house and started hosting our own servers. Having physical access to the machine (and the network HW) pretty much eliminated all of our dev ops (1). We expected the HW to fail often and such, but in 2 years the only maintenance we had to do was add more disks to the RAID array and resize the partition a few times. Even though…

> since we owned the hardware it meant that efficient utilization wasn't a requirement for us anymore This is a great point. Hyper focus on optimizing micro-containers is costly and sucks up a lot of time. You can still deploy like this in the cloud though. On AWS make a three-year reservation of a c6a.32xlarge to get 128 vCPU for $16k annually. Hetzner is cheaper, 80 vCPU for about €2,506 annually. [1] https://insta…

> On AWS make a three-year reservation of a c6a.32xlarge to get 128 vCPU for $16k annually.

plus more expensive network for cloud

Re: 11 years of hosting a SaaS

#204
post #17

Earlier quoted context omitted.

It may be a generational thing, a matter of familiarity with computing and computers. For someone who's lived through the 80s, "handrolling postgres" doesn't sound nearly as scary as you imagine. I expect the cost/benefit analysis of "handrolling postgres on a beefy Hetzner server" vs "navigating the menus and options of AWS services" would be different for different teams.

I'm looking at the ansible playbooks to setup my favourite beefy baremetal Hetzner server (128GB ram, Ryzen 9 5950X 16-Core, 450Gb fast NVME SSD, 3.5TB x2 NVME SSDs, 155€/month): - Install Debian 11 while booted in rescue mode. - Setup the root file system encryption using cryptsetup and dropbear (to enter the key during the boot through SSH). Involves chroot and some fun commands. - Setup ZFS encrypted mirror filesy…

> - Installation of (many) postgresql instances and other craps

and you missing part about fault tolerance and fall back which is most complicated.

Re: 11 years of hosting a SaaS

#205
post #172

Earlier quoted context omitted.

Yes especially data, simplicity is key as well. Where possible go with simple but abstracted cloud storage, cloud tables and then a cloud db that is managed. We use Azure mostly right now but our storage system works across Azure storage, Amazon S3, Google Cloud and others. For tables, Azure Tables mainly. For database with filtering/paging better performant and ACID compliant cloud db, CosmosDB currently which is a…

> Where possible go with simple but abstracted cloud storage This is the total opposite of simplicity. Simple is PG and backup setup. DONE. Maybe you mean "comfort"?

Managed and no need to backup with snapshots. PG still has manage concerns with size, logs, access, scale etc. Nothing you have to do with cloud storage. I do love me some PostgreSQL but use it less and less except for reporting or heavily filtered needs. Horizontal over vertical for comfort and simplicity.

Re: 11 years of hosting a SaaS

#206
post #12

> Use managed services for as long as possible Big agree here. Yes, you can save stupid money by handrolling postgres on an extremely beefy Hetzner server, or you can pay someone else and keep building your product: https://onlineornot.com/self-hosting-vs-managed-services-dec... This isn't to say, "don't bother learning how to do it yourself", but more "learn to pick your battles".

having no skin in the game, I feel this is a very interesting view, especially when you are offering a managed service building on top of another.

go through enough levels of dependencies, and someone who could do a few layers of the onion in-house can sweep in with competitive pricing/offerings a la sherlocking.

Re: 11 years of hosting a SaaS

#207
post #200
post #118

I read this article with interest, having run a SaaS for the last 8 years or so (solo-founder). I find it intriguing that the OP needed so much compute power. Computers are really powerful these days. As a counterpoint of sorts, here's my "hosting journey": * run everything on a single physical server rented at Hetzner (DE) * [... several years pass, business grows ...] * switch to ansible, learn it, spend a week or…

Just curious, if you feel comfortable sharing, what do you use as your database? I'd agree Postgres is not the right answer to every problem, but the documentation of its failure modes and the mitigations thereof make it a "good" answer to "most" problems. I'm curious what the "problem" (in access pattern terms, doesn't have to be business terms) and "solution" (i.e. persistence technology) is in your setup!

Sure. I use RethinkDB, which works well, but has been pretty much abandoned at this point. It's a good database, but it wasn't "in fashion" like many worse solutions were.

I am working on replacing it with FoundationDB. I want to have a fully distributed database with strict serializable semantics (see https://jepsen.io/consistency), and there is very little out there that gets the job done. FoundationDB is really impressive and works really well. I'm worried that it isn't "fashionable", though.

As for access patterns, I'm not sure if I understand the question, but I'll offer one thought: if you're writing an app, you don't need a "query language'. You'll quickly learn what your queries are, and the right approach is to restructure your data to fit your access patterns. Your "queries" will be written in your programming language of choice, not in the databases "query language".

I feel that the idea of a "query language" is stuck in our heads back from the days when the boss would come and tell you to produce a custom report from the database. It's just not how app databases are used these days.

Re: 11 years of hosting a SaaS

#208
post #182

Earlier quoted context omitted.

Moving data is always the trickiest part! We get a lot of Heroku customers who need help with their data migration. My honest answer is there will always be some downtime while the switch over is happening, but we know of a few ways to reduce it. Unfortunately Heroku PG databases don't allow outside replication setup, so we can't really baseline your data and then close the replication gap with a shorter downtime. Bu…

We'll keep that in mind, we're currently locked into a (ridiculous) enterprise contract with them. But it expires in November :) We're considering using this https://bucardo.org/Bucardo/ based on this https://www.porter.run/blog/migrating-postgres-from-heroku-t... Going to give it a shot when I have some free time.

Hello! Author of that little blog post there. Happy to be hit up with thoughts/feedback when you give it a shot!

Re: 11 years of hosting a SaaS

#209
post #182

Earlier quoted context omitted.

We'll keep that in mind, we're currently locked into a (ridiculous) enterprise contract with them. But it expires in November :) We're considering using this https://bucardo.org/Bucardo/ based on this https://www.porter.run/blog/migrating-postgres-from-heroku-t... Going to give it a shot when I have some free time.

Hello! Author of that little blog post there. Happy to be hit up with thoughts/feedback when you give it a shot!

Thank you so much for writing that. I was starting to lose hope of getting our data out of Heroku without some insane amount of downtime.

Re: 11 years of hosting a SaaS

#210
post #14

the progression from Heroku to AWS is very common as SaaS businesses grow. However one thing that’s missing from all “alternatives to Heroku” is what I call life after deployment or Day-2 operations. Many solutions show you a sleek demo of how easily you can deploy a Hello World app to their platform but leave you in a straight jacket when it the app is deployed and you need tools to keep it running, like changing an…

I hate what Salesforce has done to Heroku but after a year of running our saas on aws, render, and fly - here I am, back on Heroku. It sucks, but slightly less than the alternatives. Fly has potential, but they've changed/grown so much that most docs are out of date, everything is buggy, support is not very responsive, and their security posture leaves a lot to be desired. Not a fun place to be production issues pop…

(I work at Heroku) Do you have more details on what sucks? Anything we're not already tracking to fix in our public roadmap? https://github.com/heroku/roadmap/issues
Post reply on HN