The worst thing about Jenkins is that it works (2019) • Hacker News Board

Re: The worst thing about Jenkins is that it works (2019)

#161

Post by MilStdJunkie » Sun, Dec 03, 2023, 9:34 PM UTC

AceJohnny2 wrote: post #154

What would people recommend for a self-hosted CI system in 2023? I'm currently leaning towards Jenkins, because neighboring teams are using it (with some pain due to their scale, but theirs is 10x-100x ours), and I have some ~10y-old experience with it. (Our builds are compiling for embedded targets, with a Makefile-based system)

You can run a local gitlab instance, so I'd assume self-hosting as a rider on that. Unless I've been making some very bad mistakes.

GitLab's not too bad. I had a Gradle pile I had been using but once I knocked out some old Java bits I wanted to sleek my "required tools" down a bit. I've integrated some OpenProject stuff into GL also. Maybe someday . . for "minimal" clients . . I can just bring in the GitLab and call it a day, so far as server stuff is concerned.

That day is not today, however. Customers in my business want a lot - like, a LOT a lot - of project toys, and homebrewing those in GitLab is beyond my skills.

Re: The worst thing about Jenkins is that it works (2019)

#162

Post by dahart » Sun, Dec 03, 2023, 9:38 PM UTC

I’ve long had a very similar argument for why commuting in the Bay Area* sucks: because it’s just tolerable enough that people keep doing it. It’s not unmanageable, but it drains your life little by little every day.

* This, of course, applies to all long commutes everywhere, like how it applies not only to Jenkins, but lots of other software too… especially Jira.

Re: The worst thing about Jenkins is that it works (2019)

#163

Post by i386 » Sun, Dec 03, 2023, 9:42 PM UTC

maratc wrote: post #130

i386 wrote: post #79

Earlier quoted context omitted.

Thanks! My team and I made that and it was a lot of fun :)

Thanks for that, we use it all day every day! It's too bad it's been declared "end of life", without any replacement in sight, and nobody these days is fixing some bugs that are really bothering us sometimes...

I really wished we finished it and made it the default UI. Jenkins is very extensible, so there are a LOT of extension points. We certainly didn't have the right approach to the API and devex for creating react plugin extension points either.

Re: The worst thing about Jenkins is that it works (2019)

#164

Post by twitchard » Sun, Dec 03, 2023, 9:43 PM UTC

shadycuz wrote: post #144

twitchard wrote: post #140

Earlier quoted context omitted.

This is "the worst part of Jenkins is that it works". You shouldn't judge a developer tool by just what is possible to do with the tool. After all, with a little turing-completeness it is possible to do anything with anything -- you should judge the tool by what is easy to do with the tool. A good developer tool shouldn't require knowledge of a bunch of arcana to "configure correctly". A good tool protects you from "…

I wasn't judging the tools. I use several of them and with 0 complaints. I'm judging the article for being incorrect about it's specific points and for focusing on a single tool while the alternatives also suffer from the exact same issues.

You are judging Jenkins favorably because it is possible (with a bunch of arcane knowledge) to build good CI on top of it.

I am saying you should judge Jenkins disfavorably for being hard to use instead of going "skill issue" when somebody describes the pain points.

Re: The worst thing about Jenkins is that it works (2019)

#165

Post by codethief » Sun, Dec 03, 2023, 9:44 PM UTC

hardwaresofton wrote: post #55

Gitlab CI is still the best CI in the game IMO, but GitHub Actions gives it an incredible run for it's money because of how easy action re-use is. I meant to make a blog post about this, but here's a good a place as any: GitLab absolutely innovated many hard parts of CI/CD as a platform-native piece, but it feels like they lose slightly to GitHub on what GitHub does best -- social virality for developers. The problem…

> Gitlab CI is still the best CI in the game IMO,

Oh god, if this is the best CI in the game, I don't want to be part of this game anymore. I work with (as in write pipeline code for) Gitlab CI almost every day and it's absolutely horrible. Not only does YAML lack any type safety whatsoever, but every day I run into yet another open ticket in the Gitlab issue tracker because using a declarative (YAML-based) approach (as opposed to an imperative one) essentially means that the Gitlab devs need to think of every possible use case beforehand, so as to be able to define a declaration & write an implementation for it. Clearly this is impossible, so now I'm banging my head against a wall on the daily.

Re: The worst thing about Jenkins is that it works (2019)

#166

Post by chologrande » Sun, Dec 03, 2023, 9:47 PM UTC

robertlagrant wrote: post #147

chologrande wrote: post #135

Earlier quoted context omitted.

How is Gitlab CI materially different from the jenkins model? I find that the only difference is that it's YAML - so even harder to debug, and maintains the same model where you must re run an entire pipeline every commit to test functionality.

You can run locally first to test functionality : - )

oh that's nice I didn't know that! Still a stickler on yaml for code though

Re: The worst thing about Jenkins is that it works (2019)

#167

Post by MereInterest » Sun, Dec 03, 2023, 9:48 PM UTC

sevagh wrote: post #58

zdw wrote: post #16

All of the test runners have their issues. I've had as much problems with Jenkins as I've had with Github Runners and similar solutions. The best way out of this is to subdivide responsibilities: - Put all build/publish/test logic in Makefiles or scripts in the repo. This means that devs can run it locally as well. The only interaction between a test runner and the codebase should be running `make `. - Put all permis…

>- Put all build/publish/test logic in Makefiles or scripts in the repo. This means that devs can run it locally as well. The only interaction between a test runner and the codebase should be running `make `. This always breaks because then each platform has a specific way of defining env vars or secrets for those Makefiles and bash scripts. End result is devs still can't really run CI "proper" the way it's configure…

Maybe this is just me, but needing to define environment variables or secrets for testing seems like a massive problem in the test design. Does that mean the tests are dependent on the use of an external service, and that service can't be run locally?

The only part of a CI pipeline I can imagine requiring secrets would be a release/publish step. However, those would only occur after the tests run successfully, so nothing up until that point would require secrets.

Re: The worst thing about Jenkins is that it works (2019)

#168

Post by lmm » Sun, Dec 03, 2023, 9:57 PM UTC

zdw wrote: post #77

sevagh wrote: post #58

Earlier quoted context omitted.

>- Put all build/publish/test logic in Makefiles or scripts in the repo. This means that devs can run it locally as well. The only interaction between a test runner and the codebase should be running `make `. This always breaks because then each platform has a specific way of defining env vars or secrets for those Makefiles and bash scripts. End result is devs still can't really run CI "proper" the way it's configure…

This is fundamentally an interface problem. As you called out Env vars are a pretty common and well supported interface, across multiple platforms. Do you have a better option to suggest?

Set up whatever maintains your secrets (vault?) so that it works the same way in dev, CI, and real. Have whatever manages your dev versions of services you depend on (vagrant? kubernetes-docker-whateveritisthesedays?) integrate with that so that you find your service endpoints and credentials the same way in every environment.

Re: The worst thing about Jenkins is that it works (2019)

#169

Post by j1elo » Sun, Dec 03, 2023, 10:00 PM UTC

hardwaresofton wrote: post #55

Gitlab CI is still the best CI in the game IMO, but GitHub Actions gives it an incredible run for it's money because of how easy action re-use is. I meant to make a blog post about this, but here's a good a place as any: GitLab absolutely innovated many hard parts of CI/CD as a platform-native piece, but it feels like they lose slightly to GitHub on what GitHub does best -- social virality for developers. The problem…

I'm in the apparently small group of people who ignores 90% of the made & shared features, and push back against making internal ops to depend too much to any particular CI system.

We know how to build, package, and release our software, and at work I firmly support that we write those operations as standalone scripts that in principle could even run in the developer's machine itself. Moving them to CI is just changing the machine that will run the scripts, and that's where I draw the line of how much coupling to accept with any given runner: run scripts, inject secrets, and maybe accumulate artifacts between jobs.

There are a myriad of things done with Actions, such as preparing NPM, or running Docker containers. I see no point in them, you should be able to manually run the same commands, one by one, locally on a VM or empty Docker container, so at that point why not write them as a script. Use CI as a dumb shell runner. Getting to depend on all those small Actions saves 5 minutes today, only to make migrations immensely painful tomorrow.

Re: The worst thing about Jenkins is that it works (2019)

#170

Post by lmm » Sun, Dec 03, 2023, 10:02 PM UTC

lacrimacida wrote: post #15

I had a gig (circa 2014) where Jenkins was used in atrocious ways where it was basically running the show. Yet there were no problems with Jenkins itself, it was running smoothly

I once worked somewhere that had two very different Jenkins instances - one that was the typical dev CI instance, and one that was a job scheduler for all the prod batch jobs. It actually worked very well.