Jsonnet – A data templating language

We ended making the effort to simplify config of all our app’s components to just read from one large yaml file. Far happier with this solution.

For example, embedding a bash script in a templating language. This is such an incredibly crap idea I don't know how it keeps getting perpetuated. Use of this pattern works as follows: user A writes the script, tests it, shoves it into template. user B modifies the template, commits it, it later breaks silently because it was not tested, because it's in a template language, so you can't lint it and it has no tests written for it (because how did you know it would even get generated based on your input/function, let alone all the places it would be used and how, because templates). user C finds the hidden breakage after tearing out hair, removes the script from template, and writes it as a normal script that takes options and writes a test for good measure. (If you try to use normal dev patterns around the templates, like generate-template-then-commit-to-vcs, relying on and testing the generated versions, your tests have to be reworked whenever you regenerate, and later the generator gradually begins to break, so you just modify the copy in VCS rather than using the template)

A separate problem is that templating languages that give you advanced functionality lead you to want to use that functionality, and the more you do, the harder it is to maintain over time. It may not be readily apparent that trying to make Ansible configs work declaratively requires an annoying amount of complexity, or writing entire applications that cannot be easily reused and calling them "modules" or "plugins". (Whatever you are trying to template/generate, the only way to know for sure you're doing it validly is to have a function that actually parses and outputs the format according to a spec, because just spitting out a template and then externally validating it becomes very difficult to debug, because you don't know why the template is spitting out the output the way it is. So your templating language eventually becomes a compiler)

Since this is a new language, you also have to re-implement everything you normally have for a language, like a plug-in for your IDE, linter, debugger, code analyzer, port the standard libraries to different systems, etc. And of course all new devs have to learn this thing and all its quirks before they can start doing work. An entire ecosystem eventually needs to exist to support its continued use, and if you have problems, you hope this ecosystem continues to flourish so you can get support for this unsupported language tool thing, or you'll be hiring internal devs to do it.

Maybe the problem is these start out as tools for developers, and only when they get used for operations do the anti-patterns surface. I've gone through it several times and it always ends up a PITA.