Why are we templating YAML? • Hacker News Board

Re: Why are we templating YAML?

#231

Post by chriswarbo » Fri, Feb 08, 2019, 9:29 AM UTC

thanatropism wrote: post #117

pytester wrote: post #87

Earlier quoted context omitted.

XSLT was one in a litany of domain specific languages (ant, apache rewrite rules, latex macros, etc.) that evolved towards turing completeness because that's what the problem space demanded. In most if not all of these cases an existing and well designed turing complete programming language would likely have better served them.

There was a post on HN a few weeks back to the effect that it's rather easy for Turing completeness to emerge accidentally. I wish I remembered more specifics so I could find it again.

Was it http://beza1e1.tuxen.de/articles/accidentally_turing_complet... by any chance?

Re: Why are we templating YAML?

#232

Post by ekimekim » Fri, Feb 08, 2019, 9:31 AM UTC

TeMPOraL wrote: post #212

Wait, what? I feel this article is missing the bigger problem - one that for some reason just cannot die. The problem is that of gluing strings together . YAML is not an unstructured text file, it's a tree notation. Whatever "templating" or "generation" mechanism you want to use, it needs to respect the tree nature of the language it operates on. It needs to respect semantics. Gluing strings together is literally wha…

The article goes on to talk about Jsonnet, which takes the exact approach you describe - it generates JSON by aiming to be a "templated JSON" where the templating involves generating semantic objects, not strings.

Here's an example (adapted from some real-world code) where I specify the k8s cpu limit in one place, and then look up that info in several other places to avoid needing to change multiple values later:

    {
      local container = self,
      requests: {cpu: 5.5, memory: "2G"},
      limits: container.requests + {memory: "4G"},
      environment: [
        {
          name: "NUM_THREADS",
          value: std.toString(std.ceil(container.requests.cpu)),
        },
      ],
    }

Note how I can patch the container.requests object with an alternate memory limit, and how I can calculate an expression for the NUM_THREADS value in order to automatically set it to ceil() of the requested cpu.

(edited for nicer formatting of the code)

Re: Why are we templating YAML?

#233

Post by shusson » Fri, Feb 08, 2019, 9:41 AM UTC

I've recently settled on typescript object literals for creating complex config files.

Adv:

- It's still quite declarative.

- supports expressions.

- supports merging objects using the spread `...` operator. This enables breaking up large configs files into smaller files.

- supports type constraints.

- serialises to JSON easily.

Re: Why are we templating YAML?

#234

Post by aaaaaaaaaaab » Fri, Feb 08, 2019, 9:56 AM UTC

ajross wrote: post #154

rauhl wrote: post #97

Earlier quoted context omitted.

> My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. I think that you’re right, and I think it’s great, because we have a programming model in which code is data and data is code: Lisp & S-expressions. It’d be downright awesome to have a Lisp-based system which used dynamic sc…

> I think that you’re right, and I think it’s great, because we have a programming model in which code is data and data is code: Lisp & S-expressions. " Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of CommonLisp. " http://wiki.c2.com/?GreenspunsTenthRuleOfProgramming Seriously, this has happened again and again and again. You have…

And people are vehemently agreeing/disagreeing depending on their phase shift in the Turing complete vs declarative carousel.

Compare with: strongly vs weakly typed languages

Re: Why are we templating YAML?

#235

Post by gjvc » Fri, Feb 08, 2019, 10:04 AM UTC

We cannot now be far off some ansible [1]-like tool using LISP s-expressions for its configuration language.

[1] or any of that ilk

Re: Why are we templating YAML?

#236

Post by merijnv » Fri, Feb 08, 2019, 10:05 AM UTC

Cyph0n wrote: post #106

tfinch wrote: post #54

Earlier quoted context omitted.

also this is probably a nicer intro https://dhall-lang.org/

What is up with the strange comma positioning? I assume that’s just a stylistic choice?

Having prefixed commas is a rather common style in the Haskell community, because it ends up nicely matching open/closing brackets/braces and lining things up.

Since the author of Dhall comes from the Haskell community, he's kept this style.

Re: Why are we templating YAML?

#237

Post by sametmax » Fri, Feb 08, 2019, 10:06 AM UTC

Scarblac wrote: post #225

sametmax wrote: post #220

Earlier quoted context omitted.

There are several possible situations: - the django like situation: the configuration is pure code, and it's a mistake. It was not necessary, it brought plenty of problems. I wish they went with a templated toml file. - the ansible like situation: the configuration is templated static text. But with something as complex as deployment, they ended up adding more and more constructs, until they created a monstrous DSL o…

In defence of Django, the way settings.py works has been very stable for the entire lifetime of Django. It may have its problems (I don't have many issues with it) but it doesn't seem to have this problem of attracting ever more layers of abstraction on top of it. It works.

Actually, I think settings.py is not a bad idea, but it's half backed.

There should have a schema checking the setting file. There should have a better way to extend settings, and make different settings according to context, such as prod, staging or dev.

There should be a linter avoiding stupid mistakes like missing a coma in a tuple, resulting in string concatenation.

There should be variables giving you basic stuff like current dir, log dir, var dir, etc. We all make them anyway.

And there should be a better to debug the import settings problem.

But all in all, it's quick and easy to edit, and very powerful.

Re: Why are we templating YAML?

#238

Post by justincormack » Fri, Feb 08, 2019, 10:17 AM UTC

ff_ wrote: post #198

joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

This is a great analysis, but it's missing a fundamental point: why do we have a problem with these approximations of a programming language or just using a programming language to template stuff? Because your build then becomes an actual program (i.e. Turing complete) and you have to refactor and maintain it! This is the common problem of using a "programming language as configuration" (e.g. gulp?) Dhall solves exac…

This is how Lua started, as a config language, but it gradually added more features that people found useful in config, and became Turing complete.

Re: Why are we templating YAML?

#239

Post by jstrat » Fri, Feb 08, 2019, 10:23 AM UTC

okhudeira wrote: post #100

joeduffy wrote: post #57

My belief is that we've been slowly building up to using general purpose languages, one small step at a time, throughout the infrastructure as code, DevOps, and SRE journeys this past 10 years. INI files, XML, JSON, and YAML aren't sufficiently expressive -- lacking for loops, conditionals, variable references, and any sort of abstraction -- so, of course, we add templates to it. But as the author (IMHO rightfully) p…

I think what you're doing with pulumi is the right answer and it's only a matter of time before this becomes the norm. The author's examples could easily be done with plain ol' JS/ES/TS with more far more extensibility and customization when the need arises. I also feel this is where JSX got it right. Instead of creating yet-another-templating-language (looking at you Angular!), they used JavaScript and did a great j…

Crazy idea, but couldn't we use JSX for configuration?

...

Paired with Typescript, we would have the clearness of a declarative language, with the power and flexibility of a real language that is also easy to extend and navigate.

As a bonus, most tooling already exists.

Re: Why are we templating YAML?

#240

Post by tobyhinloopen » Fri, Feb 08, 2019, 10:51 AM UTC

shusson wrote: post #233

I've recently settled on typescript object literals for creating complex config files. Adv: - It's still quite declarative. - supports expressions. - supports merging objects using the spread `...` operator. This enables breaking up large configs files into smaller files. - supports type constraints. - serialises to JSON easily.

Same here... just using the same language as the rest application, usually a file with constants or a class/struct/module/whatever.

the only reason config files in another language might be required is when you require configuring the application after it is compiled to native binary code.

Obviously this doesn't apply to all these applications written in scripting languages.

I do wonder how you create type-safe config files though. I currently have a `development.ts` and a `production.ts`, where the `production.ts` is only loaded when `node_env === production`. `production.ts` contains blank/default values and the file is overwritten on the server with production secrets.