Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A skeptic's first contact with Kubernetes

blog.davidv.dev

Post reply on HN
Print view View original

91–100 of 105 posts

Re: A skeptic's first contact with Kubernetes

#91

Post by andix »

danielvaughn wrote: post #83
andix wrote: post #46

Earlier quoted context omitted.

I don't think that the k8s yamls/jsons are bad. It's just bad that we write and edit them as text files My take on the k8s yaml mess is, that we are lacking a modern replacement for Helm. The yaml templating on text file level is just pure crazyness. I think we would need an approach that is more like React/JSX or even better TSX (typescript JSX). Some possibility to get away from simple templating and more into stru…

This is a naive thought, as someone who has only been doing devops stuff for a little over a year...but what about HCL? Terraform was my favorite aspect of the devops stack, because I really enjoyed the simplicity of the configuration language. It just made sense to me.

There is a kubernetes provider for terraform: https://registry.terraform.io/providers/hashicorp/kubernetes...

I just think it doesn't map well to the declarative way of kubernetes. And HCL/Terraform often feel a bit clumsy when control flow/complex mapping is needed. Doesn't feel like a real programming language.

Terraform is great for managing cloud resources, diffing and applying changes. But this is not needed for kubernetes, in some way k8s can even be considered an alternative terraform (declare desired resources, k8s will create/update/destroy them as specified).

Re: A skeptic's first contact with Kubernetes

#92

Post by fire_lake »

mr_toad wrote: post #86
zbentley wrote: post #48

Earlier quoted context omitted.

I think it's worth re-examining the widespread opposition to writing configs in an interpreted "real" programming language, rather than a string templating language (if you're looking to split hairs, in this taxonomy PHP and JSX are real languages, but they're close to the line. Python is prototypically a real language. Awk, Jinja, CUE, and Bash/ZSH string expansion are not. If the word "real" bothers you, pick anoth…

I find Python to be one of the most useful text generation languages I’ve used. F-Strings are super useful. Partial application and list comprehension can turn pages of wallpaper code into a few short statements.

I’d encourage you to try F#. The list comprehensions are even more powerful and partial application / currying is pervasive.

Re: A skeptic's first contact with Kubernetes

#93

Post by mtndew4brkfst »

liggitt wrote: post #79
mtndew4brkfst wrote: post #12

Earlier quoted context omitted.

Kustomize is builtin to kubectl & is being removed Hadn't heard this until now, I'm a rather happy user after being jaded about Helm 2/3. Do you happen to know if this removal is sentimentally closer to "we don't want to keep maintaining kustomize" or to "we don't want to keep embedding kustomize, please use its binary directly"?

The motivation is more the latter, but it's not at all clear the proposed removal of the embedded kustomize will proceed, given the compatibility implications. See discussion at https://github.com/kubernetes/enhancements/issues/4706#issue... and following.

Thank you for this and your many many other contributions!

Re: A skeptic's first contact with Kubernetes

#94

Post by FridgeSeal »

hosh wrote: post #77
andrewaylett wrote: post #55

Earlier quoted context omitted.

I don't object to the desired state being represented in YAML, I object to generating that YAML using something that's not a "real" programming language. I don't particularly care if you generate your data structures from schemas, or generate your schemas from data structures, but I do care if you don't generate your output YAML using anything that's not type-safe. If you're generating YAML to deploy, you can do that…

I generally agree. I use Ruby for this. Duck typing is sufficient. I’ve also found that coupling manifests generation with application deployment to close a lot of doors. It’s better to modify manifests to change image references (or something similar), and have a separate process for applying manifest changes.

I’ve had great success with Flux and a small bit of CI for this.

We currently have configs in raw K8s yaml in our monorepo in kustomize base+overlay style setup. It’s super duper obvious what’s in each env, and straightforward to change them. Images are swapped in using Kustomize’s “newImage” functionality. Flux watches git and deploys to K8s.

At any point it’s super clear what the configs are, what image we were using at the time, flux just pulls in what it finds in git, so there complete separation between “make them” and “deploy them”. Nicest CICD experience ever. Zero helm tempting, zero pain.

Re: A skeptic's first contact with Kubernetes

#95

Post by icedchai »

lmm wrote: post #69
hitpointdrew wrote: post #34

Earlier quoted context omitted.

> My problem with K8s: the network abstraction layer just feels _wrong_. > I would very much prefer if K8s used public routable IPv6 for traffic delivery shudder... nothing could feel more wrong to me than public routable IPv6, yuck.

Publicly routable is wonderful. My first job was a company that happened to have somehow acquired a class B, so all our computers just had normal real addresses, they always had the same address whether you were on a VPN or a home network or whatever and remoting into the company network just worked.

Same! It was incredibly easy to obtain address space in the 80's and 90's. I have a /24 ("class C") routed to my home!

Re: A skeptic's first contact with Kubernetes

#96

Post by akvadrako »

JojoFatsani wrote: post #5

The helm yaml thing really is annoying. Unfortunately it feels like helm is too firmly embedded to unseat at this point.

You don't need helm at all. I've used k8s at a few places, big and small, and we just used kustomize or simple scripts to generate the actualized configs.

Re: A skeptic's first contact with Kubernetes

#97

Post by rockwotj »

zbentley wrote: post #48
FridgeSeal wrote: post #40

Earlier quoted context omitted.

> I've found solace in CUE We rebuilt our Kustomization layers to be built via cue, so that we could theoretically work with less mistakes. Oh my god, never again. Incredibly confusing and horrible errors. Writing new modules was painful and unclear, trying to compose them together was even worse. I still get flashbacks to that “Invalid bytes” error.

I think it's worth re-examining the widespread opposition to writing configs in an interpreted "real" programming language, rather than a string templating language (if you're looking to split hairs, in this taxonomy PHP and JSX are real languages, but they're close to the line. Python is prototypically a real language. Awk, Jinja, CUE, and Bash/ZSH string expansion are not. If the word "real" bothers you, pick anoth…

Agreed - I actually think TypeScript is easier to craft JSON output in tho: https://news.ycombinator.com/item?id=39242314

The Starlark approach is good too - I just like types if it's going to get narly and find Python typing to be suboptimal compared to TypeScript in terms of LSP and tooling (and only Facebook's Rust starlark interpreter supports types)

Re: A skeptic's first contact with Kubernetes

#99

Post by dilyevsky »

cyberax wrote: post #98
dilyevsky wrote: post #90

Earlier quoted context omitted.

How do you suggest talking to ipv4-only internet hosts and supporting ipv4-only containers?

Via the border load balancers, just like we do it now.

So basically i need to run another piece of infra that does NAT64 and DNS64 and limits my deployment options quite a bit (can't do DSR)? Totally unnecessary in cloud... Not sure how that's better for users but probably better for vendors ;)

Btw, overlay is not the only option to do CNI - Calico, Cilium and few others can do it via l3 by integrating with your equipment. Even possible in cloud but has serious scale limitations...

Re: A skeptic's first contact with Kubernetes

#100

Post by cyberax »

dilyevsky wrote: post #99
cyberax wrote: post #98

Earlier quoted context omitted.

Via the border load balancers, just like we do it now.

So basically i need to run another piece of infra that does NAT64 and DNS64 and limits my deployment options quite a bit (can't do DSR)? Totally unnecessary in cloud... Not sure how that's better for users but probably better for vendors ;) Btw, overlay is not the only option to do CNI - Calico, Cilium and few others can do it via l3 by integrating with your equipment. Even possible in cloud but has serious scale lim…

No, you misunderstand me. My dream infrastructure would run IPv6 with publicly routable IP addresses for the internal network, for everything.

IPv4 is needed only for the external IPv4 clients, and for the server code to reach any external resources that are IPv4-only. The clients are simply going to connect via the border load balancers, just as usual.

For the external IPv4-only resources, you'll need to use DNS64. But this is not any different from the status quo. Regular K8s nodes can only reach external resources through NAT anyway.

I'm actually trialing this infrastructure for my current company. We got an IPv6 assignment from ARIN, so we can use consistent blocks in US West and US East locations. We don't use K8s, though. AWS ECS works pretty great for us right now.

> Btw, overlay is not the only option to do CNI - Calico, Cilium and few others can do it via l3 by integrating with your equipment. Even possible in cloud but has serious scale limitations...

It's still an overlay network, just in hardware.

Post reply on HN