Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we templating YAML?

leebriggs.co.uk

Post reply on HN
Print view View original

201–210 of 351 posts

Re: Why are we templating YAML?

#201

Post by saagarjha »

basil-rash wrote: post #116
bamboozled wrote: post #63

Earlier quoted context omitted.

The killer feature for me over JSON is comment support, it's definitely useful to add todo comments etc.

You can just use JSON with comments though. If you have sufficient control over the technology in question to be able to completely change it to a YAML parser, surely you can change it to be a JSON+Comment parser too. See: VS Code's config files.

> You can just use JSON with comments though.

Then it's not valid JSON, and if you try to treat it as such bad things will happen.

Re: Why are we templating YAML?

#202

Post by fulafel »

ravenstine wrote: post #135
BossingAround wrote: post #27

I know I'm in a minority, but I really dislike YAML... I recently did a lot of Ansible and boy, at the beginning, I was just struggling a lot. Syntactic whitespace kills me. I don't like it in Python either, but for some reason, when I write Python, it's a lot easier. Maybe YAML is just a bit more complex (and Python has better IDE support..?)

> Syntactic whitespace kills me. Okay, I'm gonna be the asshole in the room, but how hard is it to just use consistent indentation? I can't count how many times I've heard people complain about significant whitespace in languages. Not only is it not difficult to begin with, but every code editor and IDE will show you where there's a syntax error in your YAML. People are free to dislike YAML, even for its significant…

In the YAML case: It's hard if you don't have editor support and good diagnostics. Not because you're unusually sloppy, but because you make human mistakes and because you don't know the syntax. (YAML syntax is surprisingly complex and poorly documented in the pedagogic sense). Also, the edit-debug cycle is slow with Ansible or YAML-using CI systems, so this is doubly painful.

In the Python case it's much better, because people less often casually edit .py files without editor support, and because Python has good diagnostics and it's much much harder to produce syntactically correct but semantically wrong Python by whitespace mixups.

Re: Why are we templating YAML?

#203

Post by devxpy »

joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

The problem with code as configuration is that the config file is indeterministic and it takes longer to extract information from the file.

This has long been a problem in the python/pip community, as its basically impossible for the build tools to determine the dependencies of a package without fully downloading and running the setup.py file.

Static config files are static for a reason!

Re: Why are we templating YAML?

#204

Post by makapuf »

devxpy wrote: post #203
joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

The problem with code as configuration is that the config file is indeterministic and it takes longer to extract information from the file. This has long been a problem in the python/pip community, as its basically impossible for the build tools to determine the dependencies of a package without fully downloading and running the setup.py file. Static config files are static for a reason!

Unless you import rand() your code should be deterministic. You're right about needing to run the thing to get the data (that's the point) but there is a middle ground between pure literals and fully side effects code. By example you could impose pure functions (no side effects).

Re: Why are we templating YAML?

#205

Post by hliyan »

ilovetux wrote: post #72
BossingAround wrote: post #27

I know I'm in a minority, but I really dislike YAML... I recently did a lot of Ansible and boy, at the beginning, I was just struggling a lot. Syntactic whitespace kills me. I don't like it in Python either, but for some reason, when I write Python, it's a lot easier. Maybe YAML is just a bit more complex (and Python has better IDE support..?)

I find YAML to be almost unusable. IMO it's just not intuitive. If I get to choose a format for my config files I would only use TOML, it's just better (again IMO).

This is strange. The thing I most like about YAML is how intuitive and human readable it is.

Re: Why are we templating YAML?

#206

Post by pkulak »

ilovetux wrote: post #72
BossingAround wrote: post #27

I know I'm in a minority, but I really dislike YAML... I recently did a lot of Ansible and boy, at the beginning, I was just struggling a lot. Syntactic whitespace kills me. I don't like it in Python either, but for some reason, when I write Python, it's a lot easier. Maybe YAML is just a bit more complex (and Python has better IDE support..?)

I find YAML to be almost unusable. IMO it's just not intuitive. If I get to choose a format for my config files I would only use TOML, it's just better (again IMO).

Hocon for me, if I get to choose.

Re: Why are we templating YAML?

#207

Post by pdonis »

krapp wrote: post #12
mixmastamyk wrote: post #7

Earlier quoted context omitted.

It's a lot easier for end-users to read and write by hand. If that is not a requirement, a simpler machine-friendly format is likely better.

But the simpler format of JSON should be easier to write and read.

Not when you take into account all the clutter in JSON: braces, brackets, quotes. Those are extra things to type and extra things to have to filter out when you read.

Re: Why are we templating YAML?

#208

Post by solatic »

ff_ wrote: post #198
joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

This is a great analysis, but it's missing a fundamental point: why do we have a problem with these approximations of a programming language or just using a programming language to template stuff? Because your build then becomes an actual program (i.e. Turing complete) and you have to refactor and maintain it! This is the common problem of using a "programming language as configuration" (e.g. gulp?) Dhall solves exac…

> We use it at work to generate all the Infra-as-Code configurations from a single Dhall config

This is the key bit and not something which is pitched well enough from the Dhall landing pages: using straight YAML forces you to repeat yourself in multiple areas for each Individual tool being used, and these repetitions have to stay consistent across multiple tools. What Dhall does is allow you to write a single config and use it to derive the correct configurations for each tool that you use. So you can write a single configuration file from which, eventually, every single part of your system is derived - Terraform infrastructure, Kubernetes objects, application config, everything. When you pull it off, it's simply magical.

You can think of it like this: JavaScript is a horrible, no-good, very bad language, and yet all browser programming is done in JavaScript because every browser supports it - so too, are JSON and YAML horrible configuration languages. But JavaScript gave rise to abstractions like TypeScript which are much better languages which compile down to JavaScript for compatibility. TypeScript is to JavaScript what Dhall is to JSON and YAML - the fact is, pretty much everything is configured with JSON and YAML, and Dhall makes it much, much easier to live in that world, with no need for the systems being configured to support it.

Considering the relative obscurity of Dhall, it's basically the best-kept secret in the DevOps world right now, and it's a shame more people don't know about it.

Re: Why are we templating YAML?

#209

Post by TeMPOraL »

peterwwillis wrote: post #151

YAML is a data stream, not a program. Please do not shove programs into data. Your data does not need to be "expressive", it just needs to provide input to a program. If your data files need to be complex, you need a program to generate them for you. I've danced the dance of ini -> json -> yaml -> weird hybrid -> embedded logic, and it ends with "program that asks for what the thing you want looks like and generates…

And you end up with program compiled to a config anyway, so with a proper toolchain it means your real config is the program.

People keep increasing complexity unintentionally precisely because they don't realize that code = data. There's no real distinction. Code is data is code.

You will end up having Turing completeness somewhere, it's just a matter of choosing (or blindly selecting, like most people do) where. For a popular product, it eventually gets embedded in the configuration language, turning it into half-assed programming language (see most web-related templating). For less popular products / more enterprise'y settings, you can probably get away with embedding the Turing-complete part in your bureaucracy. That is, I can't code my config to make it do what I want, but I can pay you to get developers to write some code and export it to the config language as a keyword. There's a spectrum to this, and tradeoffs galore.

But ultimately, YAML is nothing but a tree notation. Tree notation is enough to represent high-level programming languages. Lisp without parenthesis, if you might, or Python, if you squint your eyes.

Re: Why are we templating YAML?

#210

Post by igor47 »

wryun wrote: post #89
epage wrote: post #65

There have been times I've wanted to templatize my configuration but I don't want to do it with text-based templates but templates within the configuration files syntax (be it yaml, toml, or something else). Not sure what this is called, I've been calling it "structural templating". So far the only things close to this are - Azure pipeline's syntax: https://docs.microsoft.com/en-us/azure/devops/pipelines/proc... - So…

I prefer https://github.com/taskcluster/json-e Has the advantage/disadvantage that it's still valid json/yaml I wrote a scary command line wrapper for it: https://wryun.github.io/rjsone/ Has libs for Python, Go, and JS, and there's a bazel interface.

plug for json-e i found it much less painful-looking than jsonnet
Post reply on HN