Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost every infrastructure decision I endorse or regret

cep.dev

Post reply on HN
Print view View original

211–220 of 644 posts

Re: Almost every infrastructure decision I endorse or regret

#211

Post by networkchad »

macNchz wrote: post #195
tw04 wrote: post #130

Earlier quoted context omitted.

>By the time your db is big enough to cost an unreasonable amount on RDS, you’re likely a big enough team to have options. By the time your db is big enough to cost an unreasonable amount on RDS, you've likely got so much momentum that getting off is nearly impossible as you bleed cash. You can buy a used server and find colocation space and still be pennies on the dollar for even the smallest database. If you're doi…

In the small SaaS startup case, I’d say the production database is typically the most critical single piece of infra, so self hosting is just not a compelling proposition unless you have a strong technical reason where having super powerful database hardware is important, or a team with multiple people who have sysadmin or DBA experience. I think both of those cases are unusual. I’ve been the guy managing a critical…

[dead]

Re: Almost every infrastructure decision I endorse or regret

#213

Post by busterarm »

roughly wrote: post #205

The Bazel one made me chuckle - I worked at a company with an scm & build setup clearly inspired by Google’s setup. As a non-ex-Googler, I found it obviously insane, but there was just no way to get traction on that argument. I love that the rest of this list is pretty cut and dry, but Bazel is the one thing that the author can’t bring themself to say “don’t regret” even though they clearly don’t regret not using it.

I've seen Bazel reduce competent engineers to tears. There was a famous blog post a half-decade ago called something like "Bazel is the worst build system, except for all the others" and this still seems to ring true for me today.

There are some teams I work with that we'll never bother to make use Bazel because we know in advance that it would cripple them.

Re: Almost every infrastructure decision I endorse or regret

#214

Post by busterarm »

jacurtis wrote: post #135
cyounkins wrote: post #122

I've climbed the mountain of learning the basics of kubernetes / EKS, and I'm thinking we're going to switch to ECS. Kubernetes is way too complicated for our needs. It wants to be in control and is hard to direct with eg CloudFormation. Load balancers are provisioned from the add-on, making it hard to reference them outside kubernetes. Logging on EKS Fargate to Cloudwatch appears broken, despite following the docs.…

I've been running ECS for about 5 years now. It has come a long way from a "lightweight" orchestration tool into something thats actually pretty impressive. The recent new changes to the GUI are also helpful for people that don't have a ton of experience with orchestration. We have moved off of it though, you can eventually need more features than it provides. Of course that journey always ends up in Kubernetes land,…

GKE is a much better product to me still than EKS but at least in the last two years or so EKS has become a usable product. Back in like 2018 though? Hell no, avoid avoid avoid.

Re: Almost every infrastructure decision I endorse or regret

#215

Post by slyall »

sroussey wrote: post #159
paulgb wrote: post #115

Earlier quoted context omitted.

I think a lot of startups have a set of requirements that is something like: - I want to spin up multiple redundant instances of some set of services - I want to load balance over those services - I want some form of rolling deploy so that I don’t have downtime when I deploy - I want some form of declarative infrastructure, not click-ops Given these requirements, I can’t think of an alternative to managed k8s that is…

A startup with no DBA does not need redundant anything. Too small.

Places get pretty big with no dedicated DBA resources these days. Last place I was at was a Fintech SaaS with 50 engineers and half a million paying customers.

Running off a couple of medium ( $3k/month each range ) RDS databases with failover setup. ECS for apps.

Databases looked after themselves. The senior people probably spent 20% of a FTE on stuff like optimizing it when load crept up.

Place before that was a similar size and no DBA either. People just muddled though.

Re: Almost every infrastructure decision I endorse or regret

#216

Post by madduci »

morsecodist wrote: post #131

> Picking AWS over Google Cloud I know this is an unpopular opinion but I think google cloud is amazing compared to AWS. I use google cloud run and it works like a dream. I have never found an easier way to get a docker container running in the cloud. The services all have sensible names, there are fewer more important services compared to the mess of AWS services, and the UI is more intuitive. The only downside I ha…

I share your thoughts. It looks like an entire article endorsing AWS honestly

Re: Almost every infrastructure decision I endorse or regret

#217

Post by busterarm »

endisneigh wrote: post #60

Great post. I do wonder - what are the simplest K8s alternatives? Many say in the database world, "use Postgres", or "use sqlite." Similarly there are those databases that are robust that no one has heard of, but are very limited like FoundationDB. Or things that are specialized and generally respected like Clickhouse. What are the equivalents of above for Kubernetes?

The simplest k8s alternative (that is an actual alternative) is Nomad.

Re: Almost every infrastructure decision I endorse or regret

#218

Post by mr_moose »

hintymad wrote: post #73

> EKS My contrarian view is that EC2 + ASG is so pleasant to use. It’s just conceptually simple: I launch an image into an ASG, and configure my autoscale policies. There are very few things to worry about. On the other hand, using k8s has always been a big deal. We built a whole team to manage k8s. We introduce dozens of concepts of k8s or spend person-years on “platform engineering” to hide k8s concepts. We publish…

To me, it sounds like your company went through a complex re-architecturing exercise at the same time you moved to Kubernetes, and your problems have more to do with your (probably flawed) migration strategy than the tool.

Lifting and shifting an "EC2 + ASG" set-up to Kubernetes is a straightforward process unless your app is doing something very non-standard. It maps to a Deployment in most cases.

The fact that you even implemented an operator (a very advanced use-case in Kubernetes) strongly suggests to me that you're doing way more than just lifting and shifting your existing set-up. Is it a surprise then that you're seeing so much more complexity?

Re: Almost every infrastructure decision I endorse or regret

#219

Post by vander_elst »

jrockway wrote: post #71
sroussey wrote: post #51

If you are startup that can can’t afford a DBA, then why why why are you using Kubernetes?

Why wouldn't you use Kubernetes? There are basically 3 classes of deployments: 1) We don't have any software, so we don't have a prod environment. 2) We have 1 team that makes 1 thing, so we just launch it out of systemd. 3) We have between 2 and 1000 teams that make things and want to self-manage when stuff gets rolled out. Kubernetes is case 3. Like it or not, teams that don't coordinate with each other is how star…

Google has one giant codebase. I am pretty sure the aren't the only ones.
Post reply on HN