Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Almost every infrastructure decision I endorse or regret

cep.dev

Post reply on HN
Print view View original

111–120 of 644 posts

Re: Almost every infrastructure decision I endorse or regret

#111

Post by marcosdumay »

endisneigh wrote: post #60

Great post. I do wonder - what are the simplest K8s alternatives? Many say in the database world, "use Postgres", or "use sqlite." Similarly there are those databases that are robust that no one has heard of, but are very limited like FoundationDB. Or things that are specialized and generally respected like Clickhouse. What are the equivalents of above for Kubernetes?

Kubernetes aren't like that.

It's just that, you should start with a handful of backed-up pet servers. Then manually automate their deployment when you need it. And only then go for a tool that abstracts the automated deployment when you need it.

But I fear the simplest option on the Kubernetes area is Kubernetes.

Re: Almost every infrastructure decision I endorse or regret

#112

Post by kccqzy »

jrockway wrote: post #71
sroussey wrote: post #51

If you are startup that can can’t afford a DBA, then why why why are you using Kubernetes?

Why wouldn't you use Kubernetes? There are basically 3 classes of deployments: 1) We don't have any software, so we don't have a prod environment. 2) We have 1 team that makes 1 thing, so we just launch it out of systemd. 3) We have between 2 and 1000 teams that make things and want to self-manage when stuff gets rolled out. Kubernetes is case 3. Like it or not, teams that don't coordinate with each other is how star…

One giant codebase is fine. Monorepo is better than lots of scattered repos linked together with git hashes. And it doesn't really get in the way of each team managing when stuff gets rolled out.

Re: Almost every infrastructure decision I endorse or regret

#113

Post by avemg »

compumike wrote: post #69
colechristensen wrote: post #53

Earlier quoted context omitted.

PagerDuty’s cheapest plan is $21 per user month OpsGenie’s cheapest is $9 per user month but arbitrarily crippled, the plan anybody would want to use is $19 per user month So instead of a factor of ten it’s ten percent cheaper. And i just kind of expect Atlassian to suck. Datadog is ridiculously expensive and on several occasions I’ve run into problems where an obvious cause for an incident was hidden by bad behavior…

Heii On-Call is $32 per month total for your team — not per user. https://heiioncall.com/ (Full disclosure: part of the team building it)

How do you pronounce that?

Re: Almost every infrastructure decision I endorse or regret

#114

Post by janfoeh »

Almost every time I read someone's insights who works in an environment with IaaS buy-in, my takeaway is the same: oh boy, what an alphabet soup.

The initial promise of "we'll take care of this for you, no in-house knowledge needed" has not materialized. For any non-trivial use case, all you do is replace transferrable, tailored knowledge with vendor-specific voodoo.

People who are serious about selling software-based services should do their own infrastructure.

Re: Almost every infrastructure decision I endorse or regret

#115

Post by paulgb »

sroussey wrote: post #51

If you are startup that can can’t afford a DBA, then why why why are you using Kubernetes?

I think a lot of startups have a set of requirements that is something like:

- I want to spin up multiple redundant instances of some set of services

- I want to load balance over those services

- I want some form of rolling deploy so that I don’t have downtime when I deploy

- I want some form of declarative infrastructure, not click-ops

Given these requirements, I can’t think of an alternative to managed k8s that isn’t more complex.

Re: Almost every infrastructure decision I endorse or regret

#116

Post by jrockway »

kccqzy wrote: post #112
jrockway wrote: post #71

Earlier quoted context omitted.

Why wouldn't you use Kubernetes? There are basically 3 classes of deployments: 1) We don't have any software, so we don't have a prod environment. 2) We have 1 team that makes 1 thing, so we just launch it out of systemd. 3) We have between 2 and 1000 teams that make things and want to self-manage when stuff gets rolled out. Kubernetes is case 3. Like it or not, teams that don't coordinate with each other is how star…

One giant codebase is fine. Monorepo is better than lots of scattered repos linked together with git hashes. And it doesn't really get in the way of each team managing when stuff gets rolled out.

I'm a big monorepo fan, but you run into that ownership problem. "It's slow to clone"; which team fixes that?

Re: Almost every infrastructure decision I endorse or regret

#117

Post by chasd00 »

calvinmorrison wrote: post #42
sgarland wrote: post #15

Earlier quoted context omitted.

The downside is then you have many, many DBs to fight with, to monitor, to tune, etc. This is rarely a problem when things are small, but as they grow, the bad schema decisions made by empowering DBA-less teams to run their own infra become glaringly obvious.

It's because I hate databases and programming separately. I would rather slow code then have to dig into some database procdure. Its just another level of separation thats too mentally hard to manage. Its like... my queries go into a VM and now I have to worry about how the VM is performing. I wish and maybe there is a programming language with first class database support. I mean really first class not just let me r…

The language you’re talking about is APEX. I believe it comes from Oracle and is the backend language for Salesforce development. You’ll like the first class database support but that’s about it.

Re: Almost every infrastructure decision I endorse or regret

#118

Post by ezrast »

otterley wrote: post #82
jrockway wrote: post #71

Earlier quoted context omitted.

Why wouldn't you use Kubernetes? There are basically 3 classes of deployments: 1) We don't have any software, so we don't have a prod environment. 2) We have 1 team that makes 1 thing, so we just launch it out of systemd. 3) We have between 2 and 1000 teams that make things and want to self-manage when stuff gets rolled out. Kubernetes is case 3. Like it or not, teams that don't coordinate with each other is how star…

On AWS, at least, there are alternatives such as ECS and even plain old EC2 auto scaling groups. Teams can have the autonomy to run their infrastructure however they like (subject to whatever corporate policy and compliance regime requirements they might have to adhere to). Kubernetes is appealing to many, but it is not 100% frictionless. There are upgrades to manage, control plane limits, leaky abstractions, differe…

The alternatives aren't frictionless either; many items from that image are not specific to Kubernetes. I personally find AWS API's frustrating to use, so even if I were running a one-person shop (and was bound to AWS for some reason - maybe a warlock has cursed me?) I'd lean towards managing things from EKS to get an interface that fits my brain better. It's just preference, though - EC2 auto-scaling is perfectly viable if that's your jam.

Re: Almost every infrastructure decision I endorse or regret

#119

Post by jrockway »

otterley wrote: post #82
jrockway wrote: post #71

Earlier quoted context omitted.

Why wouldn't you use Kubernetes? There are basically 3 classes of deployments: 1) We don't have any software, so we don't have a prod environment. 2) We have 1 team that makes 1 thing, so we just launch it out of systemd. 3) We have between 2 and 1000 teams that make things and want to self-manage when stuff gets rolled out. Kubernetes is case 3. Like it or not, teams that don't coordinate with each other is how star…

On AWS, at least, there are alternatives such as ECS and even plain old EC2 auto scaling groups. Teams can have the autonomy to run their infrastructure however they like (subject to whatever corporate policy and compliance regime requirements they might have to adhere to). Kubernetes is appealing to many, but it is not 100% frictionless. There are upgrades to manage, control plane limits, leaky abstractions, differe…

The iceberg is fine, but using ECS doesn't absolve you from needing to care about monitoring, affinity, audit logging, OS upgrades, authentication/IAM, etc. That's generally why organizations choose to have infrastructure teams, or to not have infrastructure at all.

I have seen people rewrite Kubernetes in CloudFormation. You can do it! But it certainly isn't problem-free.

Re: Almost every infrastructure decision I endorse or regret

#120

Post by amluto »

sroussey wrote: post #44
sgarland wrote: post #30

Earlier quoted context omitted.

I think most people who’ve never experienced native NVMe for a DB are also unaware of just how blindingly fast it is. Even io2 Block Express isn’t the same.

Most databases expressly say don’t run storage over a network.

To be fair, most networked filesystems are nowhere near as good as EBS. That’s one AWS service that takes real work to replicate on-prem.

OTOH, as noted, EBS does not perform as well as native NVMe and is hilariously expensive if you try. And quite a few use cases are just fine on plain old NVMe.

Post reply on HN