Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we templating YAML?

leebriggs.co.uk

Post reply on HN
Print view View original

221–230 of 351 posts

Re: Why are we templating YAML?

#221

Post by int_19h »

Macha wrote: post #69
macspoofing wrote: post #62

Earlier quoted context omitted.

>it's as bad or worse than XML for config files XML works very well for config files. It's schema-optional (but is there), well-specified, human-readable, has plethora of supporting technologies (making things like templating easy), and is well supported by every language. At the very least it is way better than JSON.

It's missing one important part for config files. It's tedious to write by hand.

Not really. For quick authoring, configs should have pre-authored snippets for common things, that are commented out, and have adjacent descriptive comments ("Uncomment the following to ...") - this is regardless of their syntax.

And for complicated stuff, you're going to spend a lot more time reading the manual than you will actually typing those closing tags. In fact, in most cases you'd be copy/pasting bits from the manual as well.

Re: Why are we templating YAML?

#222

Post by l0b0 »

YAML is the bastard offspring of XML. A bunch of ways to write semantically identical stuff is bullshit. Let data files be data files, and build them using languages actually suited for the job. JSON is plenty complex for 105% of the use cases of YAML, with much fewer downsides.

Re: Why are we templating YAML?

#223

Post by hinkley »

TeMPOraL wrote: post #212

Wait, what? I feel this article is missing the bigger problem - one that for some reason just cannot die. The problem is that of gluing strings together . YAML is not an unstructured text file, it's a tree notation. Whatever "templating" or "generation" mechanism you want to use, it needs to respect the tree nature of the language it operates on. It needs to respect semantics. Gluing strings together is literally wha…

It’s DRY run amok. People don’t want to see the same bit of anything in two places, forgetting that code will be read, so they remove a “redundancy” but create duplicated effort for everyone, every time they have to decipher the thing later.

I also don’t think we have a workable definition of “configuration”. 70% of the config at my work is hard coded service discovery. If we moved the service discovery anywhere else (say, consul, kubernetes, hell - Docker swarm), we’d need far fewer sets of config than we have deployment environments. When there are only two or three you don’t need templating.

How often do you really have the same service deployed twice in prod and legitimately want it to work differently? I can count the scenarios I know on one hand and none of them have occurred for me in almost ten years, except read replicas and that shouldn’t be more than a few lines of config.

Re: Why are we templating YAML?

#224

Post by zoobab »

I use j2cli to template my YAMLs, pretty powerful to template stuff without using ansible. You just need to right environment variables. And you can put if logic if you want.

Re: Why are we templating YAML?

#225

Post by Scarblac »

sametmax wrote: post #220
joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

There are several possible situations: - the django like situation: the configuration is pure code, and it's a mistake. It was not necessary, it brought plenty of problems. I wish they went with a templated toml file. - the ansible like situation: the configuration is templated static text. But with something as complex as deployment, they ended up adding more and more constructs, until they created a monstrous DSL o…

In defence of Django, the way settings.py works has been very stable for the entire lifetime of Django.

It may have its problems (I don't have many issues with it) but it doesn't seem to have this problem of attracting ever more layers of abstraction on top of it. It works.

Re: Why are we templating YAML?

#227

Post by ssspaju »

I'd rephrase the question: "Why are we templating YAML with text-based templating tools?"

I and @akx wrote a templating tool called Emrichen that's specifically designed for producing YAML and JSON from YAML templates:

https://github.com/con2/emrichen

In contrast to other template systems, Emrichen templates are not just "based on YAML", they _are_ YAML. YAML tags like "!Var varname" are used to perform things like variable substitution, loops etc. Variables can be of any JSON type, not just strings, and the template is evaluated top–down.

Re: Why are we templating YAML?

#228

Post by arvinsim »

joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

> My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) points out, we just end up with a funky, poor approximation of a language.

This is the why I prefer to use a JS file for configuration instead of native JSON or YAML file if those options are available.

Re: Why are we templating YAML?

#229

Post by jrockway »

alexk wrote: post #105
jrockway wrote: post #79

I saw this title and immediately knew the article would be about Helm. I don't think anyone wants to use Helm. People use it for a set-and-forget thing that they don't care about (who cares that it's called impressive-leopard-kubernetes-dashboard, after all.) kustomize is much more sane for your own stuff: https://github.com/kubernetes-sigs/kustomize It is actually a little bit too magical for my taste, but I continu…

> I don't think anyone wants to use Helm. Here is why everyone should use Helm: Helm 2.0 introduced package as a first-class concept for Kubernetes and created the standard to distribute applications, thanks to Helm thousands of people could discover and collaborate on cloud-native deployments of the open source software https://github.com/helm/charts/tree/master/stable published and managed by organizations and cont…

That is a lot of buzzwords.

I don't really trust Helm to do anything that's actually useful in the long term. It will get something running very quickly, but whether or not it's maintainable, I am yet to be sure of. For example, very early on, I installed the helm chart for prometheus. Now I want it to live in the kube-system namespace because I am tired of seeing its resources in the default namespace. For some reason, I highly doubt that changing values.yaml to change the namespace is going to do anything other than give me a fresh instance of prometheus running in another namespace. It's not going to use the already allocated storage volume to satisfy the persistent volume claim in the new namespace. It's not going to update the other stuff in my cluster to refer to prometheus-pushgateway.kube-system.svc.cluster.local. It's not going to update my Grafana dashboards to refer to the new namespace, even though I installed Grafana with Helm! So what did I really gain? Helm isn't giving me the ability to manage the long-term lifecycle of third-party software. It just explodes some API objects all over my cluster and lets me delete most of them automatically. That's all it does.

I get why Helm is popular. You can get some piece of software running in Kubernetes with minimal effort. I would have never successfully made some random complex piece of software work correctly in Kubernetes on day 1, especially using something that assumes you deeply understand the core API objects like kustomize does. What that boils down to is that Helm doesn't go far enough, and in its current state, just encourages people to make mistakes early.

Re: Why are we templating YAML?

#230

Post by chriswarbo »

hyperpallium wrote: post #131
chriswarbo wrote: post #64

Earlier quoted context omitted.

> If you have been around long enough you still remember the world that was excited about XML and templating it using XSLT. As a hindsight it was a horrible world. I actually really like the idea behind XSLT: machine-friendly, human-tolerable, structured data + declarative rules for turning that data into a display, or a report, or whatever else. The execution was horrible though: incredibly verbose, lots of overcomp…

The syntactic mistake of XSLT was writing it in XML, XPath was a redeeming feature. Imagine if XPath was also written in XML... jq occupies the same role as XSLT, but for JSON. It can be used for templating but it's not quite as declarative as XSLT (you must pipe things through).

> The syntactic mistake of XSLT was writing it in XML, XPath was a redeeming feature. Imagine if XPath was also written in XML...

Yes, I didn't mean to imply that XPath itself is bad (although it also has to handle XML quirks like element/attribute/text, etc.).

Rather that the reason to write XSLT as XML in the first place is that it's machine readable, we can mix and match elements from different vocabularies, etc. yet most of the heavy lifting ends up as opaque string attributes :(

PS: I've done a few projects which make heavy use of jq; it's really nice, but as you say it's more of a pipeline.

Post reply on HN