Live data from Hacker News

A skeptic's first contact with Kubernetes

blog.davidv.dev

81–90 of 105 posts

Re: A skeptic's first contact with Kubernetes

#81
post #15

My problem with K8s: the network abstraction layer just feels _wrong_. It's an attempt to replicate the old model of "hard exterior, gooey interior" model of corporate networks. I would very much prefer if K8s used public routable IPv6 for traffic delivery, and then simply provided an authenticated overlay on top of it.

How would that work with load balancing and horizontal scaling?

Re: A skeptic's first contact with Kubernetes

#82
post #31
post #7

His take on text interpolation is very right. I'm a SWE turned SRE because as a developer I really enjoyed using K8s. But as a full-time SRE where I work just means YAML juggling. It's mind numbing that everybody is okay with this, this really is our domain's assembly era, albeit with whitespace, colons, dashes and brackets. I've found solace in CUE which I just run locally to catch all the small errors everybody mak…

The better way is to use manifest generators with a Turing-complete language. I wrote such a tool in Ruby. The problem is that it is written in Ruby. It's great for shops that already have Ruby expertise. Wherever I have implemented this, it's worked out great. I have my doubts about it in other shops. The community went a different way -- starting with templated JSON (not even JSON transforms). It was the least comm…

> The better way is to use manifest generators with a Turing-complete language. I wrote such a tool in Ruby.

I've seen something similar at a previous company - some Ruby DSL written to generate CloudFormation. It was used in another department though so not sure how well it worked in practice.

Re: A skeptic's first contact with Kubernetes

#83
post #46
post #7

His take on text interpolation is very right. I'm a SWE turned SRE because as a developer I really enjoyed using K8s. But as a full-time SRE where I work just means YAML juggling. It's mind numbing that everybody is okay with this, this really is our domain's assembly era, albeit with whitespace, colons, dashes and brackets. I've found solace in CUE which I just run locally to catch all the small errors everybody mak…

I don't think that the k8s yamls/jsons are bad. It's just bad that we write and edit them as text files My take on the k8s yaml mess is, that we are lacking a modern replacement for Helm. The yaml templating on text file level is just pure crazyness. I think we would need an approach that is more like React/JSX or even better TSX (typescript JSX). Some possibility to get away from simple templating and more into stru…

This is a naive thought, as someone who has only been doing devops stuff for a little over a year...but what about HCL? Terraform was my favorite aspect of the devops stack, because I really enjoyed the simplicity of the configuration language. It just made sense to me.

Re: A skeptic's first contact with Kubernetes

#84
On the last point ("Stringy Types") - k8s API types are actually defined as Protobufs[0] so they have strictly defined schemas. There are some types that are sum types (IntOrString) but generally no type depends on some other field's value afaik. Ofc that doesn't stop CRD developers from making everything a String type and then interpolating server-side (within their custom controller) based on phases of the moon and what not...

[0] - e.g https://github.com/kubernetes/api/blob/master/core/v1/genera...

Re: A skeptic's first contact with Kubernetes

#85
post #7

His take on text interpolation is very right. I'm a SWE turned SRE because as a developer I really enjoyed using K8s. But as a full-time SRE where I work just means YAML juggling. It's mind numbing that everybody is okay with this, this really is our domain's assembly era, albeit with whitespace, colons, dashes and brackets. I've found solace in CUE which I just run locally to catch all the small errors everybody mak…

Did you look at CDK8s?

Re: A skeptic's first contact with Kubernetes

#86

Earlier quoted context omitted.

> I've found solace in CUE We rebuilt our Kustomization layers to be built via cue, so that we could theoretically work with less mistakes. Oh my god, never again. Incredibly confusing and horrible errors. Writing new modules was painful and unclear, trying to compose them together was even worse. I still get flashbacks to that “Invalid bytes” error.

I think it's worth re-examining the widespread opposition to writing configs in an interpreted "real" programming language, rather than a string templating language (if you're looking to split hairs, in this taxonomy PHP and JSX are real languages, but they're close to the line. Python is prototypically a real language. Awk, Jinja, CUE, and Bash/ZSH string expansion are not. If the word "real" bothers you, pick anoth…

I find Python to be one of the most useful text generation languages I’ve used.

F-Strings are super useful. Partial application and list comprehension can turn pages of wallpaper code into a few short statements.

Re: A skeptic's first contact with Kubernetes

#87
post #15

My problem with K8s: the network abstraction layer just feels _wrong_. It's an attempt to replicate the old model of "hard exterior, gooey interior" model of corporate networks. I would very much prefer if K8s used public routable IPv6 for traffic delivery, and then simply provided an authenticated overlay on top of it.

How would that work with load balancing and horizontal scaling?

Just like it works currently. Either via dedicated load balancers or by using individual service endpoints.

Re: A skeptic's first contact with Kubernetes

#88
post #15

My problem with K8s: the network abstraction layer just feels _wrong_. It's an attempt to replicate the old model of "hard exterior, gooey interior" model of corporate networks. I would very much prefer if K8s used public routable IPv6 for traffic delivery, and then simply provided an authenticated overlay on top of it.

Something like? https://john-millikin.com/stateless-kubernetes-overlay-netwo...

It's still an overlay network.

Re: A skeptic's first contact with Kubernetes

#89
post #59
post #57

Earlier quoted context omitted.

RCL has control flow and types: https://rcl-lang.org/ . No record types yet though, so they are not yet very useful to validate e.g. a Kubernetes manifest against a schema.

Looks a lot like Jsonnet. This is going into the direction I'm talking about. Is there also some typing-checking/code-completion available for Kubernetes resources? I think this would be an essential part to improve developer experience and automatic checks. Some linter to validate the k8s manifests must be a part of a solution. It's possible to check the output, but a perfect solution would also lint the RCL source…

At the moment nothing like that exists. Eventually it should be possible to generate RCL types like https://github.com/dhall-lang/dhall-kubernetes does for Dhall.

Re: A skeptic's first contact with Kubernetes

#90
post #88

Earlier quoted context omitted.

Something like? https://john-millikin.com/stateless-kubernetes-overlay-netwo...

It's still an overlay network.

How do you suggest talking to ipv4-only internet hosts and supporting ipv4-only containers?
Post reply on HN