Live data from Hacker News

Why are we templating YAML?

leebriggs.co.uk

271–280 of 351 posts

Re: Why are we templating YAML?

#271

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

Having dealt with puppet, cloudformation, ansible and other solutions that have gone in and out of fashion and also dealing regularly with Kotlin, Java, Javascript, and recently typescript, my view is that configuration files are essentially DSLs.

DSLs ought to be type safe and type checked since getting things wrong means all kinds of trouble. E.g. with cloudformation I've wasted countless hours googling for all sort of arcane weirdness that amazon people managed to come up with in terms of property names and their values. Getting that wrong means having to dig through tons of obscure errors and output. Debugging broken cloudformation templates is a great argument against how that particular system was designed. It basically requires you know everything listed ever in the vastness of its documentation hell and somehow be able to produce thousands of lines of json/yaml without making a single mistake, which is about as likely as it sounds. Don't get me started on puppet. Very pleased to not have that in my life anymore.

On a positive note, kotlin recently became a supported language for defining gradle build files in. Awesome stuff. Those used to be written in Groovy. The main difference: kotlin is statically compiled and tools like intellij can now tell you when your build file is obviously wrong and autocomplete both the standard stuff as well as any custom things you hooked up. Makes the whole thing much easier to customise and it just removes a whole lot of uncertainty around the "why doesn't this work" kind of stuff that I regularly experience with groovy based gradle files.

Not that I'm arguing using Kotlin in place of Json/yaml. But typescript seems like a sane choice. Json is actually valid javascript, which in turn is valid typescript. Add some interfaces and boom you suddenly have type safety. Now using a number instead of a boolean or string is obviously wrong. Also typescript can do multi line strings, comments, etc. and it supports embedding expressions in strings. No need to reinvent all of that and template JSON when you could just be writing type script.

I recently moved a yaml based localization file to typescript. Only took a few minutes. This resulted in zero extra verbosity (all the types are inferred) but I gained type safety. Any missing language strings are now errors that vs code will tell me about and I can now autocomplete language strings all over the code base which saves me from having to look them up and copy paste them around. So no pain, plenty of gain.

And yes, people are ahead of me and there are actually several projects out there offering typescript support for cloudformation as well.

Re: Why are we templating YAML?

#272

File-based configs are a troublesome abstraction: they package unrelated concerns into a rigid document whose form must take a particular, application-dependent shape, and the assembly and disassembly of that document essentially becomes an API where key-value pairs are mixed with complex glue code. The application has to do this internally, but anyone who's generating their configs are also doing parts of this exter…

SQL as a configuration format is not that bad of an idea

Re: Why are we templating YAML?

#273
Maybe we should be exploring using Dhall instead of YAML.

>One of the clearest signals I’ve gotten from users is that Dhall is “the YAML killer”, for the following reasons:

>Dhall solves many of the problems that pervade enterprise YAML configuration, including excessive repetition and templating errors

>Dhall still provides many of the good parts of YAML, such as multi-line strings and comments, except with a sane standard

>Dhall can be converted to YAML using a tiny statically linked executable, which provides a smooth migration path for “brownfield” deployment.

Source: http://www.haskellforall.com/2019/01/dhall-year-in-review-20...

Re: Why are we templating YAML?

#274
post #239

Earlier quoted context omitted.

I think what you're doing with pulumi is the right answer and it's only a matter of time before this becomes the norm. The author's examples could easily be done with plain ol' JS/ES/TS with more far more extensibility and customization when the need arises. I also feel this is where JSX got it right. Instead of creating yet-another-templating-language (looking at you Angular!), they used JavaScript and did a great j…

Crazy idea, but couldn't we use JSX for configuration? ... Paired with Typescript, we would have the clearness of a declarative language, with the power and flexibility of a real language that is also easy to extend and navigate. As a bonus, most tooling already exists.

You just invented XML!

Re: Why are we templating YAML?

#275

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

There are several possible situations: - the django like situation: the configuration is pure code, and it's a mistake. It was not necessary, it brought plenty of problems. I wish they went with a templated toml file. - the ansible like situation: the configuration is templated static text. But with something as complex as deployment, they ended up adding more and more constructs, until they created a monstrous DSL o…

> it brought plenty of problems

Does anybody here personally suffered those problems that the Turing complete Django configuration creates? (I mean, not the ones caused by lack of a completness checks, or good library support, but the ones caused by too much power.)

If so, how do those problems look like?

Re: Why are we templating YAML?

#276
post #253

Earlier quoted context omitted.

Unless you import rand() your code should be deterministic. You're right about needing to run the thing to get the data (that's the point) but there is a middle ground between pure literals and fully side effects code. By example you could impose pure functions (no side effects).

That's exactly what Dhall is doing.

That's what Haskell already does. Dhall is optimizing on different dimensions (making sure the script execution ends, making the scripts verifiable at static time, making it convenient to marge files, making it convenient to centralize your configuration).

Re: Why are we templating YAML?

#277
post #178
post #172

Earlier quoted context omitted.

And I skipped toml for an object-array syntax that makes baby Jesus cry. None of these problems are hard, why are all the solutions so awful?

What are you using now?

XML, generally. I hate it but I've given up on its "successors".

Re: Why are we templating YAML?

#278
post #46

Earlier quoted context omitted.

Many language require strings to be quoted, and calling the comma requirements inane is an opinion, not an objective fact. You're not saying anything about the readability of one versus another, you're just listing gripes you have about JSON.

> Many language require strings to be quoted The languages we're comparing here are JSON and YAML. The latter does not require quoted strings, except in ambiguous cases. (And even then, actually, it technically isn't required, though it is usually the easiest thing to do.) The absence of these quotes makes the syntax get out of the way of the reader, and makes YAML comparatively easier to read. > comma requirements i…

> The latter does not require quoted strings, except in ambiguous cases.

I believe it is the right choice to always quote strings, there should be various quoting formats as no single one is perfect (normal strings with reasonable escapes,raw string, multiline-with-indentation, multiline raw strings...) and you should be able to use unquoted keys (TOML has a good balance on this problem), but unquoted strings as a default are unnecessary and problematic.

Re: Why are we templating YAML?

#279

Earlier quoted context omitted.

I still don't know how to get it to do exactly what I want. There is far too much magic involved, and experience has long demonstrated that magic is bad (Webpack confirms that belief). That being said, the concept of defining a function in, essentially, a config file seems like a step in the right direction. I don't think I'd trust that functionality outside of builds or infra-as-code, though.

What's magic about webpack? The online documentation provides quite a lot of insight into how it all fits together. It probably only seems like magic because you didn't build a fundamental understanding of how it works before using it. I use some massive webpack configurations and I understand them all quite thoroughly thanks to well-written, modularized configuration files.

For 10 years of Java/Android/Scala coding there was no need to understand how compilers combine everything into one JAR.

Re: Why are we templating YAML?

#280

I know I'm in a minority, but I really dislike YAML... I recently did a lot of Ansible and boy, at the beginning, I was just struggling a lot. Syntactic whitespace kills me. I don't like it in Python either, but for some reason, when I write Python, it's a lot easier. Maybe YAML is just a bit more complex (and Python has better IDE support..?)

Python has better IDE support and, maybe more importantly, python does not completely disallow any indentation style so it handles unsupported IDEs better. But it's essentially an IDE support problem.

My life improved a lot since I got an YAML mode for emacs. Now things would be just perfect if haskell's cabal migrated to dhall...

Post reply on HN