Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we templating YAML?

leebriggs.co.uk

Post reply on HN
Print view View original

81–90 of 351 posts

Re: Why are we templating YAML?

#81

Post by jrockway »

illumin8 wrote: post #53

His main argument is that you need to template differently for different environments (dev/stage/prod) and cloud regions (us-west/us-east/emea/apac). This is actually a solved problem, and you shouldn't be doing it in your YAML/JSON templates. You should be using an external parameter store to do this, and using a single template for everything. See https://aws.amazon.com/blogs/compute/query-for-the-latest-am... This…

Based on the appearance of Helm, this is probably referring to Kubernetes, which really does have per-environment things that can't be represented in any form other than namespaces. For example, you have an app running (that's a deployment in k8speak). You want network traffic to get to this app, so you set up a load balancer. The load balancer config needs to know the name of your app (or more precisely, a selector for "pods" that are created by the deployment), which will change between environments, so now there is that common variable that has to be updated in both places. That's a simple example but is why people are templating their k8s configs. Yes, you could work around it by giving every environment its own namespace and using the same set of objects in every environment (differing only by namespace, which should probably be in the file... but since you can't edit the files, you can pass it in to kubectl probably), but there are other cases where even that doesn't work.

You do need some way of saying "this is the base configuration and this is what we change for staging and production". Helm is a way to do that, and a popular one, but it's pretty ugly. Hence this article.

Re: Why are we templating YAML?

#82

Post by crdoconnor »

atombender wrote: post #71
BossingAround wrote: post #27

I know I'm in a minority, but I really dislike YAML... I recently did a lot of Ansible and boy, at the beginning, I was just struggling a lot. Syntactic whitespace kills me. I don't like it in Python either, but for some reason, when I write Python, it's a lot easier. Maybe YAML is just a bit more complex (and Python has better IDE support..?)

YAML is a bit bonkers in that it's a superset of JSON (all valid JSON is valid YAML), so if you don't like the whitespace sensitivity, you can write your YAML like this: { a: 42, # But you can have comments! b: "hello world", c: "and multi-line strings!", # and trailing commas! }

I wrote a pared down version of YAML because while I like the basic structure I hated the complicated bullshit like the "we also parse JSON" layered on top:

https://github.com/crdoconnor/strictyaml

Worse than JSON though, is the Norway problem.

If you remove this stuff and start validating it properly it becomes much easier to maintain.

Re: Why are we templating YAML?

#83

Post by Edmond »

epage wrote: post #65

There have been times I've wanted to templatize my configuration but I don't want to do it with text-based templates but templates within the configuration files syntax (be it yaml, toml, or something else). Not sure what this is called, I've been calling it "structural templating". So far the only things close to this are - Azure pipeline's syntax: https://docs.microsoft.com/en-us/azure/devops/pipelines/proc... - So…

we're working on this: https://www.youtube.com/watch?v=4yepPOznakk

http://codesolvent.com/config-node/

Re: Why are we templating YAML?

#84

Post by chriswarbo »

ekianjo wrote: post #66
chriswarbo wrote: post #59

Earlier quoted context omitted.

> YAML is way more easily human readable than JSON, so I started to appreciate it for readability purposes. > I guess YMMV, but after you've used both YAML and JSON for a while, you might appreciate YAML a little bit more. I've used JSON a lot, and XML and s-expressions and MessagePack and ini and YAML and a whole bunch of other formats. I usually have to fire up Google to read YAML. YAML is the only one where I rout…

a raw YAML file is readable with your eyes. A Json file needs to be prettified before you go thru it. Json is good for APIs but is not made for readibility.

We must disagree with what "readable" means then. I find JSON readable (as long as it's nicely layed out, e.g. by piping through 'jq "."'), in the sense that I can skim over the structure looking for [/]/{/}/". If I want to read some of the content, like a string, I just need to read '\"' as '"' and '\\' as '\', which is a small constant cost per (usually rare) occurrence.

With YAML it's difficult to even know the structure of what I'm looking at, due to anchors and extensions. It's also hard to discern structure from skimming, since strings can appear unquoted, and may contain unescaped lexical tokens (depending on which particular symbols it started with); hence we must carefully consider each and every character, rather than just skimming for the next token.

If I know I'm looking at a perfect YAML file, than I should be able to guess the gist of what it says, since I can make assumptions about what the syntax means. If I want to be sure, I'd be Googling for cheatsheets. Yet as a programmer, I mostly look at files when they're buggy, meaning I can't just assume that, say, an unescaped quotation mark won't terminate the string; or that a certain piece of text is allowed to run across multiple lines; or that the indentation corresponds to the nesting; etc.

Re: Why are we templating YAML?

#85

Post by jrockway »

BossingAround wrote: post #27

I know I'm in a minority, but I really dislike YAML... I recently did a lot of Ansible and boy, at the beginning, I was just struggling a lot. Syntactic whitespace kills me. I don't like it in Python either, but for some reason, when I write Python, it's a lot easier. Maybe YAML is just a bit more complex (and Python has better IDE support..?)

I don't mind YAML. I dislike that something things become strings and sometimes they become other types:

   foo: bar   # {"foo":"bar"}
   foo: "bar" # {"foo":"bar"}
   foo: 42    # {"foo":42}
   foo: "42"  # {"foo":"42"}
Other than that, no major complaints. My editor understands YAML and shows the indentation level in the background (highlight-indentation-mode) and auto-formats files so they all have consistent indentation (prettier-mode). As a result, it is not much of a nightmare to edit, despite the fact that semantic whitespace COULD cause you a lot of problems.

Re: Why are we templating YAML?

#86

Post by krferriter »

bamboozled wrote: post #63
illumin8 wrote: post #55

Earlier quoted context omitted.

I generally dislike languages like YAML or Python where whitespace matters, and can break your code, however, YAML is way more easily human readable than JSON, so I started to appreciate it for readability purposes. I guess YMMV, but after you've used both YAML and JSON for a while, you might appreciate YAML a little bit more.

The killer feature for me over JSON is comment support, it's definitely useful to add todo comments etc.

Yeah this is a huge selling point of YAML. JSON should have comments added to the spec. The other benefit to YAML is human readability, which is usually better in YAML compared to JSON. A specific glaring example of this is when there are long string-literal snippets inside the document, in YAML this is massively more readable than in JSON.

Re: Why are we templating YAML?

#87

Post by pytester »

chriswarbo wrote: post #64
miohtama wrote: post #50

If you have been around long enough you still remember the world that was excited about XML and templating it using XSLT. As a hindsight it was a horrible world. Even though YAML is not optimal, it is a human friendly compromise between too verbose XML and machine only JSON. It lacks native templating, leading to funny constructs e.g. with Ansible files. However human kind has made progress and will make progress fur…

> If you have been around long enough you still remember the world that was excited about XML and templating it using XSLT. As a hindsight it was a horrible world. I actually really like the idea behind XSLT: machine-friendly, human-tolerable, structured data + declarative rules for turning that data into a display, or a report, or whatever else. The execution was horrible though: incredibly verbose, lots of overcomp…

XSLT was one in a litany of domain specific languages (ant, apache rewrite rules, latex macros, etc.) that evolved towards turing completeness because that's what the problem space demanded.

In most if not all of these cases an existing and well designed turing complete programming language would likely have better served them.

Re: Why are we templating YAML?

#88

Post by jonreem »

joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

Also see `webpack` as a successful example of code-as-configuration in the wild.

Re: Why are we templating YAML?

#89

Post by wryun »

epage wrote: post #65

There have been times I've wanted to templatize my configuration but I don't want to do it with text-based templates but templates within the configuration files syntax (be it yaml, toml, or something else). Not sure what this is called, I've been calling it "structural templating". So far the only things close to this are - Azure pipeline's syntax: https://docs.microsoft.com/en-us/azure/devops/pipelines/proc... - So…

I prefer https://github.com/taskcluster/json-e

Has the advantage/disadvantage that it's still valid json/yaml

I wrote a scary command line wrapper for it:

https://wryun.github.io/rjsone/

Has libs for Python, Go, and JS, and there's a bazel interface.

Re: Why are we templating YAML?

#90

Post by W1ndRunner »

joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

Hi, is Pulumi an generalized AWS CDK(https://github.com/awslabs/aws-cdk/blob/master/examples/cdk-...)? Looks pretty similar :D
Post reply on HN