Almost every infrastructure decision I endorse or regret
1–10 of 644 posts
Re: Almost every infrastructure decision I endorse or regret
#2"Since the database is used by everyone, it becomes cared for by no one. Startups don’t have the luxury of a DBA, and everything owned by no one is owned by infrastructure eventually"
I think adding a DBA or hiring one to help you layout your database should not be considered a 'luxury'...
Re: Almost every infrastructure decision I endorse or regret
#3The kitchen sink database used by everybody is such a common problem, yet it is repeated over and over again. If you grow it becomes significant tech debt and a performance bottleneck.
Fortunately, with managed DBs like RDS it is really easy to run individual DB clusters per major app.
Re: Almost every infrastructure decision I endorse or regret
#4"Since the database is used by everyone, it becomes cared for by no one. Startups don’t have the luxury of a DBA, and everything owned by no one is owned by infrastructure eventually" I think adding a DBA or hiring one to help you layout your database should not be considered a 'luxury'...
Yeah I mean, hiring one person to own that for 5-10 teams is pretty cheap... Cheaper than each team constantly solving the same problems and relearning the same gotchas/operational stuff that doesn't add much value when writing your application code.
Re: Almost every infrastructure decision I endorse or regret
#5I would love to see this type of thing from multiple sources. This reflects a lot of my own experience.
I think the format of this is great. I suppose it would take a motivated individual to go around and ask people to essentially fill out a form like this to get that.
Re: Almost every infrastructure decision I endorse or regret
#6> The markup cost of using RDS (or any managed database) is worth it.
Every so often I price out RDS to replace our colocated SQL Server cluster and it's so unrealistically expensive that I just have to laugh. It's absurdly far beyond what I'd be willing to pay. The markup is enough to pay for the colocation rack, the AWS Direct Connects, the servers, the SAN, the SQL Server licenses, the maintenance contracts, and a full-time in-house DBA.
https://calculator.aws/#/estimate?id=48b0bab00fe90c5e6de68d0...
Total 12 months cost: 547,441.85 USD
Once you get past the point where the markup can pay for one or more full-time employees, I think you should consider doing that instead of blindly paying more and more to scale RDS up. You're REALLY paying for it with RDS. At least re-evaluate the choices you made as a fledgling startup once you reach the scale where you're paying AWS "full time engineer" amounts of money.
Re: Almost every infrastructure decision I endorse or regret
#7> The markup cost of using RDS (or any managed database) is worth it. Every so often I price out RDS to replace our colocated SQL Server cluster and it's so unrealistically expensive that I just have to laugh. It's absurdly far beyond what I'd be willing to pay. The markup is enough to pay for the colocation rack, the AWS Direct Connects, the servers, the SAN, the SQL Server licenses, the maintenance contracts, and a…
That's a huge instance with an enterprise license on top. Most large SaaS companies can run off of $5k / m or cheaper RDS deployments which isn't enough to pay someone. The amount of people running half a million a year RDS bills might not be that large. For most people RDS is worth it as soon as you have backup requirements and would have to implement them yourself.
Re: Almost every infrastructure decision I endorse or regret
#8> The markup cost of using RDS (or any managed database) is worth it. Every so often I price out RDS to replace our colocated SQL Server cluster and it's so unrealistically expensive that I just have to laugh. It's absurdly far beyond what I'd be willing to pay. The markup is enough to pay for the colocation rack, the AWS Direct Connects, the servers, the SAN, the SQL Server licenses, the maintenance contracts, and a…
That's a huge instance with an enterprise license on top. Most large SaaS companies can run off of $5k / m or cheaper RDS deployments which isn't enough to pay someone. The amount of people running half a million a year RDS bills might not be that large. For most people RDS is worth it as soon as you have backup requirements and would have to implement them yourself.
Definitely--I recommend this after you've reached the point where you're writing huge checks to AWS. Maybe this is just assumed but I've never seen anyone else add that nuance to the "just use RDS" advice. It's always just "RDS is worth it" full stop, as in this article.
Re: Almost every infrastructure decision I endorse or regret
#9> The markup cost of using RDS (or any managed database) is worth it. Every so often I price out RDS to replace our colocated SQL Server cluster and it's so unrealistically expensive that I just have to laugh. It's absurdly far beyond what I'd be willing to pay. The markup is enough to pay for the colocation rack, the AWS Direct Connects, the servers, the SAN, the SQL Server licenses, the maintenance contracts, and a…
You don't get the higher end machines on AWS unless you're a big guy. We have Epyc 9684X on-prem. Cannot match that at the price on AWS. That's just about making the choices. Most companies are not DB-primary.
Re: Almost every infrastructure decision I endorse or regret
#10So by and large I agree with the things in this article. It's interesting that the points I disagree with the author on are all SaaS products:
> Moving off JIRA onto linear
I don't get the hype. Linear is fine and all but I constantly find things I either can't or don't know how to do. How do I make different ticket types with different sets of fields? No clue.
> Not using Terraform Cloud No Regrets
I generally recommend Terraform Cloud - it's easy for you to grow your own in house system that works fine for a few years and gradually ends up costing you in the long run if you don't.
> GitHub actions for CI/CD Endorse-ish
Use Gitlab
> Datadog Regret
Strong disagree - it's easily the best monitoring/observability tool on the market by a wide margin.
Cost is the most common complaint and it's almost always from people who don't have it configured correctly (which to be fair Datadog makes it far too easy to misconfigure things and blow up costs).
> Pagerduty Endorse
Pagerduty charges like 10x what Opsgenie does and offers no better functionality.
When I had a contract renewal with Pagerduty I asked the sales rep what features they had that Opsgenie didn't.
He told me they're positioning themselves as the high end brand in the market.
Cool so I'm okay going generic brand for my incident reporting.
Every CFO should use this as a litmus test to understand if their CTO is financially prudent IMO.