Live data from Hacker News

How I think about Kubernetes

garnaudov.com

31–40 of 68 posts

Re: How I think about Kubernetes

#33
On the use of GitOps for k8s, I think it makes sense for application workloads, and less sense for raw infrastructure definitions (unless you are running at such a scale that your infrastructure is often scaled like an application).

For my infrastructure definition repo, I will apply it in my terminal with kubectl, watch, and then merge the PR/commit to master. I often need to do this progressively just to roll back if I see resource consumption or other issues, it would be quite dangerous to let the CI pipeline apply everything and then for me to try and change declarations whilst the control plane API is totally starved for resources.

Also (and maybe this is me not doing "proper devops", I don't care), I will often want to tinker a bit with the declaration, trying a bunch of little changes, and then commiting once all is satisfactory. That "dev loop" is less productive if I have to wait for a CI pipeline for every step.

Re: How I think about Kubernetes

#34

I always wonder if things can be simpler. When you think of a really simple DB you think of SQLite. What's the really simple K8s? Even doing a single node deployment these days seems complicate with Prometheus, Grafana, etc. etc. docker/podman compose up with quadlets and all of this stuff just seems so eh. I really like the idea of something like Firebase, but it never seems to work out or just move the complexity t…

One of the issues is that the open source Helm charts (or whatever) for something like Grafana do not come out-of-the-box with good config. I spent a significant amount of time a little while ago reading blogs to get Grafana to use up to date indexing algorithms, and better settings, etc.

Considering these companies make money when you use their hosted solution, this is not surprising, and it just goes to show TANSTAFL.

Re: How I think about Kubernetes

#35

> Thinking of Kubernetes as a runtime for declarative infrastructure instead of a mere orchestrator results in very practical approaches to operate your cluster. Unpopular opinion, but the source of most of the problems I've seen with infrastructures using Kubernetes came from exactly this kind of approach. Problems usually come when we use tools to solve things that they weren't made for. That is why - in my opinion…

Kubernetes is explicitly designed to do what the article describes. In that respect the article is just describing what you can find in the standard Kubernetes docs.

> it is super important to treat a container orchestrator a container orchestrator.

Which products do you think are only “container orchestrators”? Even Docker Compose is designed to achieve a desired state from a declarative infrastructure definition.

Re: How I think about Kubernetes

#36
post #13

Earlier quoted context omitted.

It would have helped if you tell us why you don’t like this approach.

It's right there: > the source of most of the problems I've seen with infrastructures using Kubernetes came from exactly this kind of approach But some more concrete stories: Once, while I was on call, I got paged because a Kubernetes node was running out of disk space. The root cause was the logging pipeline. Normally, debugging a "no space left on device" issue in a logging pipeline is fairly straightforward, if th…

You mention a questionably designed custom operator and an add-on from a SIG. This is like blaming Linux for the UI in Gimp.

Re: How I think about Kubernetes

#37
post #36

Earlier quoted context omitted.

It's right there: > the source of most of the problems I've seen with infrastructures using Kubernetes came from exactly this kind of approach But some more concrete stories: Once, while I was on call, I got paged because a Kubernetes node was running out of disk space. The root cause was the logging pipeline. Normally, debugging a "no space left on device" issue in a logging pipeline is fairly straightforward, if th…

You mention a questionably designed custom operator and an add-on from a SIG. This is like blaming Linux for the UI in Gimp.

Also not like logging setups outside of k8s can't be a horror show too. Like, have you ever had to troubleshoot a rsyslog based ELK setup? I'll forever have nightmares from debugging RainerScript mixed with the declarative config and having to read the source code to find out why all of our logs were getting dropped in the middle of the night.

Re: How I think about Kubernetes

#38

On the use of GitOps for k8s, I think it makes sense for application workloads, and less sense for raw infrastructure definitions (unless you are running at such a scale that your infrastructure is often scaled like an application). For my infrastructure definition repo, I will apply it in my terminal with kubectl, watch, and then merge the PR/commit to master. I often need to do this progressively just to roll back…

This more direct interaction by using kubectl happens on a development cluster? Or is it on the same that runs production workloads?

Re: How I think about Kubernetes

#39
I sometimes joke that Kubernetes is a mass experiment in teaching people how to write Go via YAML.

The giant nested YAML you come across is the input (pre-deserialization)/output (post-serialization) for the declared types:

https://github.com/kubernetes/api/blob/master/core/v1/types....

Fortunately, or unfortunately, I am the only person that finds humor in this.

Re: How I think about Kubernetes

#40
post #20

Earlier quoted context omitted.

yeah but why?

Over-engineering to solve a problem that doesn't exist, thereby making one.

sure…what’s the motivation? is this seriously that difficult of a question to answer? this “solution” was put in place for a reason, what’s the reason? does nobody actually know?
Post reply on HN