Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is Kubernetes adoption so hard?

plural.sh

Post reply on HN
Print view View original

11–20 of 21 posts

Re: Why is Kubernetes adoption so hard?

#11

Post by dpc_01234 »

It's a marketing post, but since you and me are here already:

Replace YAML with proper "general purpose" featureful configuration language like Nix (or a typed version of it: Nickel). You could do Dhall or Jsonnet, but IMO, Nix is the way to go and keeps gaining traction.

Abandon Dockerfiles, and let the containers be described in the same general purpose language (see https://nixos.wiki/wiki/NixOS_Containers ), that the cluster can build and handle the same way.

Have the cluster itself be described in the same configuration language.

Wrap it all the proper document library APIs, so the most common use-cases, are a handful of code, with ability to expand to lower level details once it needed. Hey, did I mention that we have a general purpose configuration language that makes it possible?

At the end it should be possible to `git init .`, write 10 lines of Nix code (import stdlib, import 1 node standard cluster, run a servive with hello-world application), call `deploy-this` with your cloud access key and get a k8s cluster with a hello world application behind a TLS.

Then when you decide you need more nodes, more services, you just add new lines, or rewrite existing high level ones into more detailed calls.

All of this currently required gluing tons of things together (Docker, Terraform, kops, Helm, Kustomize, ArgoCD and what not) all using different programming languages, xmls, jsons, yamls, HCLs, Dockerfiles and so and so.

Re: Why is Kubernetes adoption so hard?

#12

Post by fancythat »

I am going to state a somewhat unpopular opinion, Kubernetes in their current form are just short from unusable in complex production environment. In order to run a full Kubernetes stack, you need to get a multitude of components running, each with own name with unknown effects on the underlying system.

I have experiences with two forms of deployment: one was an on-prem installation two years ago, on which one of the 13 Java 8 applications had very large latencies when accessing Oracle DB, otherwise working fine when it was deployed on simple VM. All of those applications had been done with the same DB logic, and we couldn't find the issue on our own, so we asked third-party to debug this issue for us: they couldn't pin point the problem, even with commercial tools. Their answer was just, something is off with your Kubernetes installation and that was it.

My second, on-going experience, is my current assignment with Fortune 500 company, that uses GKE for running hundreds of nodes, after migrating from on-prem VMs. Almost every other week (99% reliability - yeah right), some part of the system just dies and leaves services unreachable or unresponsive. There is a continuous effort to solve this issues and even Google support was contacted with the answer boiling down to: shit happens, deal with it. The only solution in those situations is either to have alarms go off so that Ops can restart something, or just to wait until everything comes back up again on its own.

The whole ecosystem was a good idea that lacks proper tool and stability to provide substantial benefits over the bunch of VMs, IMO.

Re: Why is Kubernetes adoption so hard?

#14

Post by citrin_ru »

fancythat wrote: post #12

I am going to state a somewhat unpopular opinion, Kubernetes in their current form are just short from unusable in complex production environment. In order to run a full Kubernetes stack, you need to get a multitude of components running, each with own name with unknown effects on the underlying system. I have experiences with two forms of deployment: one was an on-prem installation two years ago, on which one of the…

I have an impression that K8s to some extent benefits from a positive feedback loop - developers/ops advocate it usage to learn and put it on their CV (even if otherwise it is a bad choose), managers choose it thinking that it would be easy to hire people with K8s experience given that a lot of people around trying to learn it. I could not tell about my experience with K8s (NDA) but it doesn't look pretty to me too.

Re: Why is Kubernetes adoption so hard?

#15

Post by notmypenguin »

K8s and the like will never replace coding for actual computers nor do they replace the many layers they sit upon. Networking isn’t well thought out, and we generally need to stop propagating the idea that this is the future of application execution, it’s a hack, a clever hack, but insufficient to obviate hypervisors and operating systems. If you truly seek shrinkwrapped application s look into exokernels.

Re: Why is Kubernetes adoption so hard?

#16

Post by zerbinxx »

60secs wrote: post #8

Because helm charts lack a schema. Programming by yaml configuration means you spend time searching github for an example kind of like what you're trying to do instead of having actual tools which allow auto-completion. Editing helm charts it the ultimate form of throwing spaghetti at the wall and praying it works, without being able to actually test anything locally.

I know this may be controversial, but ChatGPT is an absolute godsend for this exact problem. K8s YAML an overly flexible markdown language (itself with infinite options that you may or may not choose to RTFM about), complex custom resources that may or may not be Done Right (wrt abstraction, readability, general best practices, unsurprising-ness), and multiple biomes worth of competing tools and possible applications, it’s fairly safe to say that, like natural language, the entire world (of k8s) is overdetermined and suffers badly from Yet Another Framework/Tool overload. Generative AI, thankfully, is really good at at least giving your workable examples of some of the features, and can make up for my own lack of interest in RTFM at 9 on Sunday night when stage goes down and we have barbarous developers eager to push their Rube Goldberg machines to production bright and early tomorrow morning.

Re: Why is Kubernetes adoption so hard?

#17

Post by bzzzt »

fancythat wrote: post #12

I am going to state a somewhat unpopular opinion, Kubernetes in their current form are just short from unusable in complex production environment. In order to run a full Kubernetes stack, you need to get a multitude of components running, each with own name with unknown effects on the underlying system. I have experiences with two forms of deployment: one was an on-prem installation two years ago, on which one of the…

> Almost every other week (99% reliability - yeah right), some part of the system just dies and leaves services unreachable or unresponsive. There is a continuous effort to solve this issues and even Google support was contacted with the answer boiling down to: shit happens, deal with it.

The entire point of Kubernetes is redundancy through multiple stateless service instances that can and will be killed at any moment. If you take an application that doesn't work in such an environment, for instance a highly stateful application, that will cause pain. If you want simple 'lift and shift' to the cloud avoid Kubernetes.

Re: Why is Kubernetes adoption so hard?

#18

Post by vilunov »

dpc_01234 wrote: post #11

It's a marketing post, but since you and me are here already: Replace YAML with proper "general purpose" featureful configuration language like Nix (or a typed version of it: Nickel). You could do Dhall or Jsonnet, but IMO, Nix is the way to go and keeps gaining traction. Abandon Dockerfiles, and let the containers be described in the same general purpose language (see https://nixos.wiki/wiki/NixOS_Containers ), that…

> Nix is the way to go and keeps gaining traction

Hard to take this seriously since nix-lang is the largest (but not the only) roadblock to nix adoption in my observations. Besides, it hardly provides any benefits other than an even more exotic syntax.

Re: Why is Kubernetes adoption so hard?

#19

Post by vbezhenar »

belmont_sup wrote: post #10
60secs wrote: post #8

Because helm charts lack a schema. Programming by yaml configuration means you spend time searching github for an example kind of like what you're trying to do instead of having actual tools which allow auto-completion. Editing helm charts it the ultimate form of throwing spaghetti at the wall and praying it works, without being able to actually test anything locally.

At this point, if it’s painful enough, why isn’t compiling-to-yml tools more popular? Example: https://github.com/dhall-lang/dhall-kubernetes Haven’t used dhall myself but I’d definitely prefer a DSL on top of yaml.

I use plain yamls and kustomize. It works for me. Not ideal, but definitely not on the level to replace it with the whole another language.

Yamls do have a scheme, editors can autocomplete it.

I've heard that helm is painful. Never used it.

Re: Why is Kubernetes adoption so hard?

#20

Post by fancythat »

bzzzt wrote: post #17
fancythat wrote: post #12

I am going to state a somewhat unpopular opinion, Kubernetes in their current form are just short from unusable in complex production environment. In order to run a full Kubernetes stack, you need to get a multitude of components running, each with own name with unknown effects on the underlying system. I have experiences with two forms of deployment: one was an on-prem installation two years ago, on which one of the…

> Almost every other week (99% reliability - yeah right), some part of the system just dies and leaves services unreachable or unresponsive. There is a continuous effort to solve this issues and even Google support was contacted with the answer boiling down to: shit happens, deal with it. The entire point of Kubernetes is redundancy through multiple stateless service instances that can and will be killed at any momen…

Applications in question are all stateless and worked in distributed VM environment without issues, usually with simple Nginx or HAProxy redundancy setup. I see your point and it is valid, however, this wasn't the case in the examples I mentioned.
Post reply on HN