Live data from Hacker News

Why are we templating YAML?

leebriggs.co.uk

291–300 of 351 posts

Re: Why are we templating YAML?

#291

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

This is my experience and it's clearly biased from maybe one bad example but ... Scons is an example of code over configuration and from what I could tell I never met someone that truly understood it. Because it was code over configuration every programmer added their own interpretation of what was supposed to happen, no programmer truly understood what was really going one and it turned into one giant mess of trying to understand different programmers hacks and code to get the build to work. I'm sure some Scons expert will tell me how I'm full of crap but I'm just saying, that's my experience.

So, what's my point? My point is configuration languages help in that they push "the one true way" and help to enforce it. Sure there are times you end up having to work around the one true way but given very powerful tools of a full language for configuration leads to chaos or at least that's my experience. Instead of being able to glance at the configuration and understand what's happening because it follows the one true way you instead end up with configuration language per programmer since every programmer will code up stuff a different way.

Re: Why are we templating YAML?

#292
Reading the commentary on whitespace, my thoughts immediately jumped to the C preprocessor. Even though it's built into the language, it has the same sorts of problems as a template engine has with generating YAML: the preprocessor just wasn't enough aware of the syntactic structure of the language to make it easy to generate anything of significant complexity.

I'm not proud of this (and like to think I could come up with something better these days), but this code was a bit of a nightmare for that reason:

https://github.com/mschaef/vcsh/blob/master/vm/number.c#L256

Lisp macros do better, but they have the problem that the macros (and their potentially unusual evaluation rules) can easily just blend in with ordinary function calls.

Re: Why are we templating YAML?

#293

Earlier quoted context omitted.

> If you have been around long enough you still remember the world that was excited about XML and templating it using XSLT. As a hindsight it was a horrible world. I actually really like the idea behind XSLT: machine-friendly, human-tolerable, structured data + declarative rules for turning that data into a display, or a report, or whatever else. The execution was horrible though: incredibly verbose, lots of overcomp…

The syntactic mistake of XSLT was writing it in XML, XPath was a redeeming feature. Imagine if XPath was also written in XML... jq occupies the same role as XSLT, but for JSON. It can be used for templating but it's not quite as declarative as XSLT (you must pipe things through).

I quite liked SXML + SXSLT back in the day (scheme syntax)

Re: Why are we templating YAML?

#294
post #153

Earlier quoted context omitted.

Is there a reason http://opsmop.io/ and http://vespene.io/ are both down and their Github's both say "DISCONTINUED"? https://github.com/opsmop/opsmop https://github.com/vespene-io/vespene

Looks like it happened a few days ago: https://threadreaderapp.com/thread/1091710068234641408.html

> In this case, lots of discussions show everyone is busy, has no time, and also ... increasingly they have interest in low-code/no-code type solutions. This is not open source as whole, just the IT ops vertical

Looks like he doesn’t believe the code approach is viable as much as other people are claiming in this thread.

Re: Why are we templating YAML?

#295
post #251
post #97

Earlier quoted context omitted.

> My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. I think that you’re right, and I think it’s great, because we have a programming model in which code is data and data is code: Lisp & S-expressions. It’d be downright awesome to have a Lisp-based system which used dynamic sc…

As the kids say: stop trying to make Lisp happen, it's not going to happen. It has become yet another community that's fighting a struggle that everyone else ended years ago, like the few Japanese in jungles who refused to surrender. I'm not entirely sure why it's not been adopted, but I suspect it's because most people strongly prefer (a) visually semantically different scope delimiters and (b) function-outside-brac…

Oh my! So web-assembly is not 'happening' then ? May it REST in peace.

Re: Why are we templating YAML?

#296

Earlier quoted context omitted.

There are several possible situations: - the django like situation: the configuration is pure code, and it's a mistake. It was not necessary, it brought plenty of problems. I wish they went with a templated toml file. - the ansible like situation: the configuration is templated static text. But with something as complex as deployment, they ended up adding more and more constructs, until they created a monstrous DSL o…

> it brought plenty of problems Does anybody here personally suffered those problems that the Turing complete Django configuration creates? (I mean, not the ones caused by lack of a completness checks, or good library support, but the ones caused by too much power.) If so, how do those problems look like?

Now that you say it, it's true I didn't have problems with too much power.

I never had an untrusted party editing my config, nor did I use data from any.

Also, you can make the same mistakes in the setting file that in any code file, but it's not more or less important.

In fact, all the problem I had could have been solved by better integration: solving the import problem, making composition easy, adding checks, allow loading data from several sources and merge them, presenting them in a unify interface.

If I'm being honest, problem with settings.py may have not been that it's Python, but that it's a flat file with no strong conventions, tooling or best practices.

I could raise the issue that you can't read the config from another language, but I never had to, and good tooling would allow a synced export or an API to consume the settings.

Same for writing, or live settings.

Re: Why are we templating YAML?

#297

Earlier quoted context omitted.

Everything is hard when you don't have editor support and good diagnostics. Don't blame YAML because you prefer to use Notepad.exe However, missing/extra whitespace is not "hard". You would be docked points in an English paper and you should be docked points as a programmer. So, whitespace aside... Tell me what is easier to edit without built-in syntax support: JSON, or YAML? If we define "easy" as "how long it takes…

I see you restate your argument for clarity, let me try the same :) 1) YAML is a configuration file format, and it's targeting user groups and environments where people use ad hoc terminal based or os-bundled editors, such tools being nano or Notepad, and such users being sysadmins for example. 2) YAML implementations (=parsers) have poor diagnostics compared to Python, separate from the editor issue, and 3) YAML syn…

In my opinion no one should be using Notepad for programming work or configurations that are more than 1-2 dozen lines. Nano is about the same: It's a text editor with no inherent tooling for configuration files and syntax support.

A construction worker can't complain that nails are hard to use because they showed up to work with a baseball bat. Or that they're designed badly because they brought a soft aluminum hammer with a tiny head instead of one made with a stronger metal and large impact surface. Tooling is important. Vim and several graphical editors have syntax support. Notepad++ if you're on Windows.

> YAML syntax is more prone than Python to parsing correctly but producing unwanted semantics when you make a mistake.

If you made a mistake, you made a mistake. Why do you expect a program with a mistake to work correctly? Use tooling which prevents you from making mistakes. And the particulars of YAML semantics are orthogonal to how your editor handles it. Yes = True, No = False, etc., for better or worse, but that's got nothing to do with your editor.

> many/most people editing YAML files don't know YAML syntax very well compared to this scenario. If their knowledge of English was at the same level, misplaced whitespace would not be chief of their problems in a graded English paper.

I wholeheartedly agree. So if a programmer complained to me that they were having issues related to inconsistency with whitespace, I would be suspect of their general programming abilities and would start reading their code to determine if the problem lies deeper than just getting an extra space here or there: Incorrect tooling, linting, sloppiness, inattention to detail... All of these things get in the way of well-written software.

As for whitespace in general, and the fact that it's harder for linters to determine and highlight if a block is correctly scoped without enclosures... Python and others have this same issue.

> It is of course a structurally valid (philosophically consistent) argument that people should not make mistakes and they should suffer when they do, but this goes generally against the consensus of configuration language usability thinking.

True, and I agree. Everyone makes mistakes even with things as simple as rote data entry. This is why tooling is incredibly important.

Tightrope-walking at great heights is incredibly dangerous. Practitioners accept this danger. They typically wear harnesses to mitigate the danger of falling. Of course, some people like to live on the edge and set records involving no harnesses. If someone like Dean Potter fell while walking a tight-rope freeform with no harness and plunged to their death, their last thought wouldn't be, "Shit, I knew that tightrope was poorly designed and dangerous," it will be "Shit, I wish I'd been wearing a harness."

We can't remove our harnesses and then complain that mistakes are too frequent and costly.

Re: Why are we templating YAML?

#298

Earlier quoted context omitted.

There are several possible situations: - the django like situation: the configuration is pure code, and it's a mistake. It was not necessary, it brought plenty of problems. I wish they went with a templated toml file. - the ansible like situation: the configuration is templated static text. But with something as complex as deployment, they ended up adding more and more constructs, until they created a monstrous DSL o…

After years of working with cfengine then ansible I finally went to a bespoke BSD ports work alike with optional client/server and json configuration components. Never looked back.

What does it look like ?

Re: Why are we templating YAML?

#299

Earlier quoted context omitted.

Everything is hard when you don't have editor support and good diagnostics. Don't blame YAML because you prefer to use Notepad.exe However, missing/extra whitespace is not "hard". You would be docked points in an English paper and you should be docked points as a programmer. So, whitespace aside... Tell me what is easier to edit without built-in syntax support: JSON, or YAML? If we define "easy" as "how long it takes…

Editing JSON is ok without specific format support, it just looks like any other C-like language. Editing YAML is basically impossible without specific support, your editor will almost certainly break any file you open and destroy relevant information on the process.

Care to elaborate? Your statement is hollow on its own.

How does your editor destroy information? What kind of information is destroyed? Why is your editor rearranging bits in validly encoded text files?

Re: Why are we templating YAML?

#300
post #249

Earlier quoted context omitted.

Not a user of Dhall, just a fan, but refactoring of Dhall configuration should be extremely easy. You make a change, and your configuration stays the same, which is easy to verify. (Thanks to https://en.wikipedia.org/wiki/Normalization_property_(abstra... ) For TC languages, comparing if two programs (original and refactored) do the same thing is not solvable in general. If the language is not TC then it is more feas…

You can compare the outputs of two programs. Sure, a TC program may not finish to produce output you can compare, but in my experience that's only a theoretical problem.

You can do more than just compare the output of two programs in Dhall. You can verify using a semantic integrity check that two programs are the same for all possible inputs. For example:

  $ dhall hash 
The cryptographic hash is smart enough that many behavior-preserving changes don't perturb the hash.
Post reply on HN