What's the break-even number of machines you must manage before Kubernetes starts to make sense?
A skeptic's first contact with Kubernetes
61–70 of 105 posts
Re: A skeptic's first contact with Kubernetes
#62Great writeup on the core fundamentals, saved this to share with engineers who are new to k8s and need a quick primer. Re: This piece - > Given the Controller pattern, why isn't there support for "Cloud Native" architectures? > I would like to have a ReplicaSet which scales the replicas based on some simple calculation for queue depth (eg: queue depth / 16 = # replicas) > Defining interfaces for these types of events…
See KEDA and Karpenter for advanced k8s acaling
Karpenter is great for managing spot fleet nodes as well. Most of our clusters run a small aws managed node group for karpenter and the rest of the nodes will be spot fleet and managed by karpenter.
Re: A skeptic's first contact with Kubernetes
#63What's the break-even number of machines you must manage before Kubernetes starts to make sense?
Kubernetes starts to make sense when you don't know the number of machines you need (and therefore desire auto-scaling based on system load).
Re: A skeptic's first contact with Kubernetes
#64His take on text interpolation is very right. I'm a SWE turned SRE because as a developer I really enjoyed using K8s. But as a full-time SRE where I work just means YAML juggling. It's mind numbing that everybody is okay with this, this really is our domain's assembly era, albeit with whitespace, colons, dashes and brackets. I've found solace in CUE which I just run locally to catch all the small errors everybody mak…
KCL is amazing and growing quickly.
Can use Crossplane function KCL too.
Re: A skeptic's first contact with Kubernetes
#65Earlier quoted context omitted.
I've not yet gone down that path, in which other ways does your workload need to adapt? I understand that you may want to get more value out of logs, and include side-cars, you may want inter-pod encryption and include service meshes, etc; but that'd be something that requires extra complexity in any setup
OIDC integration, RBAC, data persistence/replication across nodes, observability, mTLS. And yes, you're right, all these things are complex in any situation, except when you simply use a load balancer and two servers. There are companies estimated to be worth close to US$2B using with sub 30 servers using a service you may have heard of: Stack Overflow ( https://nickcraver.com/blog/2016/03/29/stack-overflow-the-ha...…
None of those things are specific to Kubernetes, though. If anything it's a great forcing function to do the things you aught to for non-k8s deployments. It's far easier to write, say, a systems unit for a service that is properly configurable, has health checks, etc.
Re: A skeptic's first contact with Kubernetes
#66His take on text interpolation is very right. I'm a SWE turned SRE because as a developer I really enjoyed using K8s. But as a full-time SRE where I work just means YAML juggling. It's mind numbing that everybody is okay with this, this really is our domain's assembly era, albeit with whitespace, colons, dashes and brackets. I've found solace in CUE which I just run locally to catch all the small errors everybody mak…
The better way is to use manifest generators with a Turing-complete language. I wrote such a tool in Ruby. The problem is that it is written in Ruby. It's great for shops that already have Ruby expertise. Wherever I have implemented this, it's worked out great. I have my doubts about it in other shops. The community went a different way -- starting with templated JSON (not even JSON transforms). It was the least comm…
Sort of. Kubernetes is lowest common denominator by design. Specific ecosystems had much more efficient solutions to these problems (e.g. rather than having to deploy a whole docker container to do a version upgrade you deploy just your specific web application) but the problem was those solutions only worked for those specific ecosystems.
Re: A skeptic's first contact with Kubernetes
#67His take on text interpolation is very right. I'm a SWE turned SRE because as a developer I really enjoyed using K8s. But as a full-time SRE where I work just means YAML juggling. It's mind numbing that everybody is okay with this, this really is our domain's assembly era, albeit with whitespace, colons, dashes and brackets. I've found solace in CUE which I just run locally to catch all the small errors everybody mak…
Something curious I noticed: while this comment was posted "6 hours ago", when I click reply, it instead says that the comment was posted "3 days ago".
Is this happening to others as well?
From digging around it looks like the comment really was posted 3 days ago, looking at the `title="2024-07-29T08:41:07"` attribute in the timestamp tag.
Re: A skeptic's first contact with Kubernetes
#68Earlier quoted context omitted.
> I've found solace in CUE We rebuilt our Kustomization layers to be built via cue, so that we could theoretically work with less mistakes. Oh my god, never again. Incredibly confusing and horrible errors. Writing new modules was painful and unclear, trying to compose them together was even worse. I still get flashbacks to that “Invalid bytes” error.
I think it's worth re-examining the widespread opposition to writing configs in an interpreted "real" programming language, rather than a string templating language (if you're looking to split hairs, in this taxonomy PHP and JSX are real languages, but they're close to the line. Python is prototypically a real language. Awk, Jinja, CUE, and Bash/ZSH string expansion are not. If the word "real" bothers you, pick anoth…
How are you drawing the line to put Dhall in your bad list and Starlark in your good list? As far as I can see they're extremely close cousins.
Re: A skeptic's first contact with Kubernetes
#69My problem with K8s: the network abstraction layer just feels _wrong_. It's an attempt to replicate the old model of "hard exterior, gooey interior" model of corporate networks. I would very much prefer if K8s used public routable IPv6 for traffic delivery, and then simply provided an authenticated overlay on top of it.
> My problem with K8s: the network abstraction layer just feels _wrong_. > I would very much prefer if K8s used public routable IPv6 for traffic delivery shudder... nothing could feel more wrong to me than public routable IPv6, yuck.
Publicly routable is wonderful. My first job was a company that happened to have somehow acquired a class B, so all our computers just had normal real addresses, they always had the same address whether you were on a VPN or a home network or whatever and remoting into the company network just worked.
Re: A skeptic's first contact with Kubernetes
#70[flagged]