51–60 of 97 posts
Earlier quoted context omitted.
I think the only exit to this cycle is something like dhall, which adds features like functions and imports, but is "total" / forbids side-effects. https://dhall-lang.org/
YTT is pretty nice too and basically glues star lark and yaml together.
Is this typical? All of our Gitlab CI files are well under 100 lines. What sorts of things are these pipelines doing that require so much configuration?
Our CI steps are basically:
* Build
* Run some static analysis
* Test
* Publish build artifacts
With each step taking only a few lines. Most of the “heavy lifting” is managed by other tools like npm, or some scripts we have checked into the project, and our CI process just kicks off those steps.
Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…
I never understood these project where somebody makes a very easy declarative thing and makes it imperative.
Programming in Python is order of magnitude "harder" and error prone than writing a simple YAML file.
Another step in the endless cycle of configuration vs code. It's not an accident that we are in a deep cycle of constrained configuration languages (yaml/json/etc etc). People chose to go there because before that we had a cycle of using programming languages and people hated it, for all sorts of good reasons. Now I see we are on the way back into adding wrappers around the static config files, to turn them back into…
The big difference is that config files are, at some point after interpolation and templating, static data structures composed of primitive data types. Further, these configs usually act as declarative languages, giving directives to pipelines, controllers, etc., typically with idempotent side-effects. This is just good software engineering. You have some static data structure which describes some result you want to…
The important part is that the interface has to have the same semantics as a config file: atomic/transactional. For example the builder pattern does this. You can enjoy all the expressiveness and power of the high-level language, but at some point you create the final config/pipeline/job-graph, call whatever needs to be called, and the provider of the interface needs to validate that.
That's why the semantics of Terraform is better than Ansible's.
It seems like a lot of people don't understand that having a DECLARATIVE language/configuration/whatever such a huge advantage it's insane. It's so easy to write, you can never make logical mistakes, avoid all kind of bugs, it's just easy to learn and you can just write the desired result. I never understood these project where somebody makes a very easy declarative thing and makes it imperative. Programming in Pytho…
For a simple config, sure.
But as soon as you have something more complicated, you end up with:
- a frankenstein monster DSL, with pseudo flow control + funcs backed in through a patched up templating system
- weird error messages, no stack trace, and type errors that accumulates
- no DRY
- no tooling: type checks, linting, de bugger, logger. Forget about it.
Case in point: ansible playbooks.
So unless your config is going to stay under 20 lines, just use a real programming language. If you don't want to use Python, fine. Use dhall, nix, jsonet or something else.
> We started this project because of Gitlab CI yaml files growing over thousands of lines. Is this typical? All of our Gitlab CI files are well under 100 lines. What sorts of things are these pipelines doing that require so much configuration? Our CI steps are basically: * Build * Run some static analysis * Test * Publish build artifacts With each step taking only a few lines. Most of the “heavy lifting” is managed b…
What I would really like to see is a CI system that lets me write a script in a language of my choice instead of defining a pipeline config file. That way I can run the pipeline locally, put breakpoints in, etc. Nuke [1] gets close but there are still a lot of tasks that don't have C# bindings, such as publishing build artifacts and uploading test results. While I'm dreaming about my perfect CI, I'd also like the abi…
Every tool I've used allows you to do this. Just write the script in the language of your choice, then run it in the pipeline.
A pipeline should be considered like "markup" around your scripts. Everything should work independently and on any system. The pipeline config just tells the scheduler which order to run it in, which bits can run in parallel, what artifacts to keep etc.
Earlier quoted context omitted.
YTT is pretty nice too and basically glues star lark and yaml together.
via comments ! YAML is an abomination in itself ( spaces for logic?!), the last thing it needed was comments for extra logic.
Earlier quoted context omitted.
via comments ! YAML is an abomination in itself ( spaces for logic?!), the last thing it needed was comments for extra logic.
Proper spacing makes code readable. Forcing it is a good thing.