Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What we learned after a year on Kubernetes

about.gitlab.com

Post reply on HN
Print view View original

41–50 of 162 posts

Re: What we learned after a year on Kubernetes

#41

Post by __float »

iso8859-1 wrote: post #30
SomaticPirate wrote: post #25

Earlier quoted context omitted.

Dhall continues to a be a glimmer of hope in this space. I agree that helm caused the entire space to distort. There is so little reusability in charts. Dhall seems to be the start of reversing this, but I hesitate to mention it since the ergonomics are still rough. If YAML is the assembly language of k8s, Dhall looks like what you would write the compiler in.

Given that the Dhall type system supports things that the other configuration languages do not, isn't it very hard to migrate to for a project like k8s? I don't see how they could support something in parallel to Dhall, and still enable all the features Dhall provides.

Dhall does have many features that can still "compile down" to YAML or JSON (or some others, too). Depending on this target language, some Dhall features are unavailable.

I don't think OP meant Kubernetes is changing to Dhall, but that we'll see something like Dhall on top of Kubernetes YAML/JSON that catches on.

Re: What we learned after a year on Kubernetes

#43

Post by Aeolun »

gscho wrote: post #17
AaronFriel wrote: post #9

Earlier quoted context omitted.

Are you familiar with Pulumi? It does away with the notion that you need serialized yaml files, although in its current incarnation it can produce them, and instead runs programs which generates collections of named (and hierarchically defined) resources which can then be "applied" as a batch. It uses lazy values to generate a directed graph to determine the order for creating and tearing down resources as well as sa…

...and you need to use their SaaS to make it work, no?

Nope, you can sync state using just an S3 bucket (or local filesystem). It’s a bit hidden in their docs though.

Re: What we learned after a year on Kubernetes

#44

Post by kqr »

dpc_pw wrote: post #39
aeontech wrote: post #33

Earlier quoted context omitted.

Have you looked at Dhall or Tanka? They are both designed to solve exactly this problem (and I couldn’t agree with you more)

Honestly, I am not a fan of "configuration languages". Why do we need another bunch of languages. Were the existing languages not good enough? There's already shitload of them. The only place where I find it useful is when the config is provider by an untrusted party. But even then scheme, lua would probably fit the bill. Not strong feelings about it though, maybe I'm wrong. All I want is a library for my favorite la…

I don't know about the others, but at least Dhall provides features that are literally impossible to get from a general purpose programming language, like semantic hashing, totality, and such.

Re: What we learned after a year on Kubernetes

#45

Post by theptip »

ishcheklein wrote: post #3

We are using k8s as well for a project that we also provide on-prem install too. We decided to use kustomize for now vs doing helm for now. Curious, have someone had experience with both to compare. What are the benefits of using helm?

Helm is good for importing other components (eg Redis).

I find it pretty tedious for authoring my own Yaml specs; any time you need to do some sort of composition you have to write templates, and then mess with indentation operators, because significant white space. Templating yaml is a pretty bad idea in my experience.

As of Helm3 the most egregious part (Tiller) is gone, so it’s more recommendable.

Scaffold lets you write (roughly) native k8s yaml files and layer them together. IDEs know the schema for this yaml at this point, so you get code completion on all your input files. This breaks in Helm. Scaffold is good for simple apps, and falls down a bit if you need to dynamically configure your specs (eg render a parameterized manifest for each review app). The programming model for scaffold is a bit hard to grok, based on my experience of ramping up a team on it. Not rocket science just a bit unintuitive at first.

Re: What we learned after a year on Kubernetes

#46

Post by sandGorgon »

justicezyx wrote: post #7
dpc_pw wrote: post #4

Earlier quoted context omitted.

`helm` is an absolute garbage. I'll reserve judgment about `kustomize` until I have more practical experience, but so far it looks to me that it's going to be another YAML disaster. The problem with k8s ecosystem is that developers conflated k8s using yaml for a semi-human-readable serialization of k8s resources, as reason to employ YAML to absolutely everything. What I'd like is to generate serialized yaml files ...…

Insider joke: K8s was born to solve the same problem Borg solved or simply design in such a way to not have that problem. One of them is the config files hell. Borgcfg files ranks #3 in terms of line count across all of Google. (Borgcfg language is similar to Jsonnet, helm) In terms of the amount of power wielded buy a single CLI, `borgcfg`, this little program that has less than 50k lines of nontesting code, stands…

What do you think about Cue ? There's a very interesting issue about why Cue improves upon Borg.

https://github.com/cuelang/cue/issues/33#issuecomment-483615...

Re: What we learned after a year on Kubernetes

#47

Post by justicezyx »

sandGorgon wrote: post #46
justicezyx wrote: post #7

Earlier quoted context omitted.

Insider joke: K8s was born to solve the same problem Borg solved or simply design in such a way to not have that problem. One of them is the config files hell. Borgcfg files ranks #3 in terms of line count across all of Google. (Borgcfg language is similar to Jsonnet, helm) In terms of the amount of power wielded buy a single CLI, `borgcfg`, this little program that has less than 50k lines of nontesting code, stands…

What do you think about Cue ? There's a very interesting issue about why Cue improves upon Borg. https://github.com/cuelang/cue/issues/33#issuecomment-483615...

Sorry, I have little knowledge on Cuelang.

@mpvl is one of my most admired Googler. I promised to contribute to the Cue project, but haven't found time (typical startup employee situation, but it seems there is no trouble to find time surfing hacker news...).

Re: What we learned after a year on Kubernetes

#48

Post by alephu5 »

Nullabillity wrote: post #26
creztoe wrote: post #14

Earlier quoted context omitted.

Could you explain why helm is garbage? I think it suits its purpose rather well without being too complex. You can essentially "plug-in" different types of resources rather easily. Especially in v3 now that you don't need to install Tiller and can avoid setting those cluster permission requirements. Have you tried some Kubernetes api libraries? You can generate and configure resources with [python kubernetes-client](…

> Could you explain why helm is garbage? Not the OP, but.. 1. YAML string templating makes it very easy to get indentation and/or quotation wrong, and the error messages can easily end up pretty far from the actual errors. Structured data should be generated with structured templating. 2. "Values" aren't typechecked or cleaned. 3. Easy to end up in a state where a failed deploy leaves you with a mess to clean up by h…

It's not going to solve all your problems but dhall can fix your first few gripes. I've been using it for several months and it's an excellent way to write configuration imo.

Re: What we learned after a year on Kubernetes

#49

Post by zelphirkalt »

I do have only very minimal xp in k8s, as I have tried to use it only once (and never again). To me it seem like mOst of the comments are about the unusable and unmaintainable yaml configuration files. This seems like exactly the issue I had a few years ago, when I was cursing, because I could not find a spec or docs for the yaml stuff in all the pages and pages of seemingly important documentation and thus could not figure out how to transform a docker only deployment into k8s deployment. I also remember having to set up helm, according to whatever tutorial or official guide I was following. It was all a distant nightmare, that I do not wish to repeat.

Re: What we learned after a year on Kubernetes

#50

Post by 0x008 »

creztoe wrote: post #14
dpc_pw wrote: post #4

Earlier quoted context omitted.

`helm` is an absolute garbage. I'll reserve judgment about `kustomize` until I have more practical experience, but so far it looks to me that it's going to be another YAML disaster. The problem with k8s ecosystem is that developers conflated k8s using yaml for a semi-human-readable serialization of k8s resources, as reason to employ YAML to absolutely everything. What I'd like is to generate serialized yaml files ...…

Could you explain why helm is garbage? I think it suits its purpose rather well without being too complex. You can essentially "plug-in" different types of resources rather easily. Especially in v3 now that you don't need to install Tiller and can avoid setting those cluster permission requirements. Have you tried some Kubernetes api libraries? You can generate and configure resources with [python kubernetes-client](…

I love how the kubernetes client is only compatible with python <= 3.6
Post reply on HN