Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jenkins pipelines as YAML

jenkins.io

Post reply on HN
Print view View original

51–60 of 76 posts

Re: Jenkins pipelines as YAML

#51

Post by sciurus »

dunckr wrote: post #20
ruffrey wrote: post #16

I am so grateful for this. As a non-Java person, using the Groovy pipelines was incredibly painful to incrementally fix on the server. Nothing like pushing 20 commits in a row and forcing a build, only to find each time that an odd syntax error.

I had the same problem. Luckily jenkins provides an API which you can validate your jenkinsfile. I setup the deploy to always lint before attempting to run. https://jenkins.io/doc/book/pipeline/development/#linter

That only works for declarative pipelines, not scripted ones, though.

Re: Jenkins pipelines as YAML

#53

Post by jdi92 »

scarface74 wrote: post #32
Chico75 wrote: post #12

This is just lowering themselves to the same level as every other CI tool. For sure YAML is an easier barrier to entry to overcome, but it's also a low ceiling. YAML doesn't let you transcribe logic easily compared to any programming language, and build logic always tends to go up. Programming languages also allows you to create abstraction to remove some of the complexity from the users.

My experience with yml based builds and deployments is basically using the yml file to tell the build system what code to run to do the build - usually other shell scripts or some other scripting language that can also be in the same git repo.

I read yaml for ansible like an ordered array.

It’s a namespace with the values you want inline, in my head

They’re INI files or .cfg to me

Not code

The code is the Python in Ansible, or Go in TF, that applies those values

Using config mgmt tools like they’re vanilla programming languages is weird.

Re: Jenkins pipelines as YAML

#54

Post by rubenbe »

rhencke wrote: post #24
zdw wrote: post #3

Looks interesting, although it seems very similar to another project, Jenkins Job Builder ( https://docs.openstack.org/infra/jenkins-job-builder/ ) that can describe jobs as YAML or JSON, and is designed one level higher as a meta-jobs creator that can create many jobs all with slight variations, with all of the config kept in version control (no manual fiddling with the Jenkins GUI, other than troubleshooting). With…

Also highly recommended is Jenkins Job DSL, which, despite also using Groovy like Jenkins Pipeline, takes a far different approach. In Job DSL, you write Groovy scripts to declare your job. These scripts, in turn, write the config.xml that Jenkins uses to actually define and execute jobs. One of the big benefits to us is that it is _not_ an alternate execution engine like Jenkins Pipeline is. It just takes over the j…

Having worked with both, it is also interesting to know that Job DSL is aware of jobs that have to be removed, while Job builder is not. That was one of the reasons we chose Job DSL as our configuration method.

Re: Jenkins pipelines as YAML

#55

Post by singingfish »

stefan_ wrote: post #17
ruffrey wrote: post #16

I am so grateful for this. As a non-Java person, using the Groovy pipelines was incredibly painful to incrementally fix on the server. Nothing like pushing 20 commits in a row and forcing a build, only to find each time that an odd syntax error.

Still have to do that to build the thing in the first place. I do not understand at all how every CI solution out there makes it entirely impossible to trivially test a pipeline. Here we are building something whose entire point is that it can run wherever , but the one place it can't run is my local machine? It's like they are doing it out of spite.

Hah. I have a short perl script that trivially tests individual components of gitlab pipelines (parses the yaml, extracts the script components, saves them to a temp file then executes the temp file as bash script). The code is short but a little contorted. Works well though. Will be releasing into the wild in a couple of months.

Re: Jenkins pipelines as YAML

#56

Post by zorkw4rg »

merinowool wrote: post #23

There are technologies that are build using COD - consultant oriented development. That is one of the principals is that only a trained consultant would be able to configure it and that would require knowledge impossible to find in freely available documentation. It also needs multitude of gotchas that couldn't be solved using common sense. Jenkins I believe is one of such tools. We have completely moved away from it…

I guess rather than having to deal with the agony to move away from a sufficiently large and complex Jenkins setup, people rather delude themselves in that its actually good software.

Just the amount of obtuse setup that thing requires is crazy and always ends in a mess. I have no idea why there seem to be so many people praising that thing in this thread. I don't fault Jenkins though, its just old obsolete software that should be replaced.

Re: Jenkins pipelines as YAML

#57

Post by epage »

rhencke wrote: post #24
zdw wrote: post #3

Looks interesting, although it seems very similar to another project, Jenkins Job Builder ( https://docs.openstack.org/infra/jenkins-job-builder/ ) that can describe jobs as YAML or JSON, and is designed one level higher as a meta-jobs creator that can create many jobs all with slight variations, with all of the config kept in version control (no manual fiddling with the Jenkins GUI, other than troubleshooting). With…

Also highly recommended is Jenkins Job DSL, which, despite also using Groovy like Jenkins Pipeline, takes a far different approach. In Job DSL, you write Groovy scripts to declare your job. These scripts, in turn, write the config.xml that Jenkins uses to actually define and execute jobs. One of the big benefits to us is that it is _not_ an alternate execution engine like Jenkins Pipeline is. It just takes over the j…

We've been using jobdsl for years.

We've been moving away from people authoring jobdsl. Instead we have a custom format suited to our policies. We still use jobdsl for taking care of knowing the configuration details of all of the plugins we use.

We have a custom format for

- Decoupling server configuration (version, plugin versions) from build configuration. This is especially important because jobdsl regularly breaks compatibility. You can't just check your build description into your repo and live on a happy life.

- Reuse a lot of build steps across our different repos. We have a lot of common steps specific to use and we wanted to reduce our boiler plate.

I am looking into moving off of jobdsl to jjb. In theory, jjb supports generating for multiple plugin versions. As a step towards aligning two distinct build farms, we want our custom format supported on both. Unfortunately, our build farms are on different versions of plugins. I could have our tool support generating for different jobdsl versions but am instead looking at relying on jjb's ability to do this.

Re: Jenkins pipelines as YAML

#58

Post by oweiler »

KaiserPro wrote: post #42
Chico75 wrote: post #12

This is just lowering themselves to the same level as every other CI tool. For sure YAML is an easier barrier to entry to overcome, but it's also a low ceiling. YAML doesn't let you transcribe logic easily compared to any programming language, and build logic always tends to go up. Programming languages also allows you to create abstraction to remove some of the complexity from the users.

"This is just lowering themselves to the same level as every other CI tool" you mean making a stable simple and repeatable interface that can be saved in source control?

Groovy based declarative pipelines were also version-controlled, stable and repeatable.

Re: Jenkins pipelines as YAML

#59

Post by vorg »

humbleMouse wrote: post #39
rhencke wrote: post #24

Earlier quoted context omitted.

Also highly recommended is Jenkins Job DSL, which, despite also using Groovy like Jenkins Pipeline, takes a far different approach. In Job DSL, you write Groovy scripts to declare your job. These scripts, in turn, write the config.xml that Jenkins uses to actually define and execute jobs. One of the big benefits to us is that it is _not_ an alternate execution engine like Jenkins Pipeline is. It just takes over the j…

Agreed, I love the groovy jenkins job DSL. It's so awesome and simple. Easy to put everything on github and forget about it.

Besides all the special cases in the normal use of Apache Groovy (e.g. definition of `==` different to Java), you also have to remember all the Jenkins-specific special cases (e.g. Groovy's collections methods don't work). I wouldn't call it simple or easy at all.

Re: Jenkins pipelines as YAML

#60

Post by michaelneale »

lomnakkus wrote: post #41
twblalock wrote: post #36

Earlier quoted context omitted.

> At my current gig, again an enterprise, it is Jenkins everywhere. They do the most complex things with it, orchestrating entire releases, integration tests, etc. You just answered your own question -- people use Jenkins because it can do all of those things.

It's possible to use Jenkins to do all of those things. It's also enormously costly in terms of manpower, etc. It's truly absurd just how much shepherding Jenkins requires once you have build slaves, etc. The pipeline stuff is heaps better, but unfortunately it doesn't really work for the workflow we're using in our shop, so... sad faces all round :(.

Yes it is harder than it should be - there is a project under way to try to make this config and plugin pain go away: https://jenkins.io/blog/2018/04/06/jenkins-essentials/ (not sure if that name will stick, will maybe just be Jenkins at some point in the future). Hopefully that will help one day, and be far more efficient use of time.
Post reply on HN