You will never agree 100% with someone else when it comes to decisions like this, but clearly there is a lot of history behind these decisions and they are a great starting point for conversations internally I think.
Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
51–60 of 247 posts
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#52It's weird that one of the reasons that you endorse AWS is that you had regular meetings with your account manager but then you regret premium support which is the whole reason you had regular meetings with your account manager.
As a counterpoint, I find our AWS super team to be a mix of 40% helpful, 40% “things we say are going over their head,” 20% attempting to upsell and expand our dependence. It’s nice that we have humans but I don’t think it’s a reason to choose it or not. GCP’s architecture seems clearly better to me especially if you are looking to be global. Every organization I’ve ever witnessed eventually ends up with some kind of…
Similar to my experience with the two. We didn't have regular meetings with our GCP account manager, but they did help us and we had a technical support rep there we were in contact with sometimes. We rarely heard from anyone at AWS, and a friend had some horror stories of reporting security issues to AWS.
Architecturally I'd go with GCP in a heartbeat. Bigquery was also one of the biggest wins in my previous role. Completely changed out business for almost everyone, vs Redshift which cost us a lot of money to learn that it sucked.
You could say I'm biased as I work at Google (but not on any of this), but for me it was definitely the other way around, I joined Google in part because of the experience of using GCP and migrating AWS workloads to in.
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#53Earlier quoted context omitted.
SRE here who has dealt with this before. Everything in article is excellent point but other big point is schema changes become extremely difficult because you have unknown applications possibly relying on that schema. It's also at certain point, the database becomes absolutely massive and you will need teams of DBAs care and feeding it.
Not only will you need a team of DBAs caring for it, but you'll never be able to hire them.
No organization I have seen prioritizes a DBA's requirements, concerns, or approach. They certainly don't pay them enough to deal with that bullshit, so I was out.
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#54I see you regret Datadog but there's no alternative - did you end up homebrewing metrics, or are you just living with their insane pricing model? In my experience they suck but not enough to leave.
VictoriaMetrics stack. Better, cheaper, faster queries, more k8s native, etc.
Easy to run with budget saved from not being on Datadog + attracts smart and observability minded engineers to your team.
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#55It's weird that one of the reasons that you endorse AWS is that you had regular meetings with your account manager but then you regret premium support which is the whole reason you had regular meetings with your account manager.
As a counterpoint, I find our AWS super team to be a mix of 40% helpful, 40% “things we say are going over their head,” 20% attempting to upsell and expand our dependence. It’s nice that we have humans but I don’t think it’s a reason to choose it or not. GCP’s architecture seems clearly better to me especially if you are looking to be global. Every organization I’ve ever witnessed eventually ends up with some kind of…
Global VPCs are very nice but they feel like a single blast radius.
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#56> Picking Terraform over Cloudformation: Endorse I, too, prefer McDonald's cheeseburgers to ground glass mixed with rusty nails. It's not so much that I love Terraform (spelled OpenTofu) as that it's far and away the least bad tool I've used in the space.
Terraform/openTofu is more than OK. The fact that you can use to to configure your Cisco products as well as AWS is honestly great for us. It's also a bit like ansible: if you don't manage it carefully and try to separate as much as possible early, it starts bloating, so you have to curate early. Terragrunt is the only sane way to deploy terraform/openTofu in a professional environment though.
We can also use expect to configure Cisco routers and AWS infrastructure, doesn’t mean we should.
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#57> Picking Terraform over Cloudformation: Endorse I, too, prefer McDonald's cheeseburgers to ground glass mixed with rusty nails. It's not so much that I love Terraform (spelled OpenTofu) as that it's far and away the least bad tool I've used in the space.
I've been very happy using cdk for interacting with aws. Much better than terraform and the like.
I second this. I do use some terraform, but for most of our stacks, CDK has been fantastic.
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#58There is a lot of "stuff" for liking to keep it simple. Great article though!
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#59Earlier quoted context omitted.
Any opinion on Pulumi?
My opinion is there are not enough good software developers doing DevOps, and HCL is simple enough and can have pretty good guardrails on it. My biggest concern is people shooting themselves in the foot because the static analysis tools available for HCL don't work with Pulumi.
It’s an unfortunate truth that good software developers aren’t crazy enough to want to do it.
Re: Infrastructure decisions I endorse or regret after 4 years at a startup (2024)
#60> Picking Terraform over Cloudformation: Endorse I, too, prefer McDonald's cheeseburgers to ground glass mixed with rusty nails. It's not so much that I love Terraform (spelled OpenTofu) as that it's far and away the least bad tool I've used in the space.
CDK is far better than Terraform.