Jenkins pipelines as YAML

YAML has the odor of TLS when the winds are just right. Not to be tech religious, but over-engineered, complex, wordy requirements tend to be expensive to implement, secure, audit and support. Tome-like specifications like YAML, PDF and TLS are invitations to bugs and security vulnerabilities. I remember printing some 800+ pages of the early Adobe PDF spec from around 2002, it was a container format with multiple encodings, able to store JavaScript, JScript, executable code and more... and thinking “this is going to lead to no end of malware and most of these dangerous features will need to be disabled or removed.”

JSON might not be the most expressive, flexible, precise or concise text data encoding, but it’s sure darn easy to parse with a solid, simple spec. HCL and TOML tend to be better choices for ease of mechanized configuration management.

Also highly recommended is Jenkins Job DSL, which, despite also using Groovy like Jenkins Pipeline, takes a far different approach.

In Job DSL, you write Groovy scripts to declare your job. These scripts, in turn, write the config.xml that Jenkins uses to actually define and execute jobs.

One of the big benefits to us is that it is _not_ an alternate execution engine like Jenkins Pipeline is. It just takes over the job of writing the job XML for you, instead of editing it through the GUI.

We had a large, complex, manually 'versioned' job chain that did not lend itself to being re-written in Jenkins Pipeline, as it made heavy use of plugins that were not supported by Pipeline at the time.

By using Job DSL, we were able to incrementally replace the manually maintained jobs with script-generated ones. Importantly, with nothing breaking for end users! All download links, URLs, etc of the new jobs exactly matched the old ones, so it was a very painless, incremental adoption that people did not even notice!

If you've got an existing job chain you are trying to get under control, Job DSL comes highly recommended.

Don't get me started on stages and concurrency, or trying to figure out where an error came from.

I'm wondering, who uses Jenkins nowadays and why?

A few years ago, at one of my first gigs in an enterprise environment, I used Jenkins for the first time to test and build my stuff. When I needed a newer version of my compiler or specific linter, some fellow had to install that on the VM that was running Jenkins.

Later, working in a more modern environment, we started using Gitlab CI. It was a bliss. I specified the docker image with my favourite tooling and my stuff got built in there. When some tooling changed, I updated my image.

At my current gig, again an enterprise, it is Jenkins everywhere. They do the most complex things with it, orchestrating entire releases, integration tests, etc. I don't know what to think of this yet.

How does the HN crowd see this?

I’m by no means a devops expert and while I have used Jenkins before, the only times I’ve had to design a CI/CD system from scratch I used either Microsoft’s hosted version of TFS (with git) -VSTS or AWS’s Code Pipeline.

With AWS Codepipeline, you can also specify a custom Docker image with your build environment and all of the tools you need - it’s in fact required if you want to build on Windows with the .Net framework (not .Net Core) and it’s also orchestrated with yml files.

I much prefer that approach to Jenkins.