What would people recommend for a self-hosted CI system in 2023? I'm currently leaning towards Jenkins, because neighboring teams are using it (with some pain due to their scale, but theirs is 10x-100x ours), and I have some ~10y-old experience with it. (Our builds are compiling for embedded targets, with a Makefile-based system)
Because not every company wants to allow their source code to the cloud. Basically all the reasons organization self host their code repositories (and a bunch do) all apply.
The worst thing about Jenkins is that it works (2019)
261–270 of 275 posts
Re: The worst thing about Jenkins is that it works (2019)
#262Earlier quoted context omitted.
Random example: I want to define a setup job to build & push a base Docker image to our container registry. I then want to use this image as IMAGE in all subsequent jobs. This is impossible because the IMAGE field in YAML cannot be dynamic (determined at runtime) but I'd like to version/tag my image using the $CI_COMMIT_SHA.
This is definitely possible. We've been building an image in one phase and then running it in subsequent phases for years with gitlab. I think you are going about this wrong. Are you generating an image tag dynamically? When you are tagging the image, make sure that you generate the tag deterministically based on information that is available to gitlab when the pipeline is created. So for example, you could use the t…
Re: The worst thing about Jenkins is that it works (2019)
#263Earlier quoted context omitted.
Random example: I want to define a setup job to build & push a base Docker image to our container registry. I then want to use this image as IMAGE in all subsequent jobs. This is impossible because the IMAGE field in YAML cannot be dynamic (determined at runtime) but I'd like to version/tag my image using the $CI_COMMIT_SHA.
The image may be determined at runtime, but it’s not required to exist until a runner picks up the job. So use the $CI_COMMIT_SHA in the image name and push the image in a job that runs before the other jobs that use the image. You might also want to look into Downstream Pipelines.
The issue is that the IMAGE field in YAML doesn't pick up environment variables that are being passed on from a previous job via a dotenv artifact.
As for downstream pipelines, please see https://news.ycombinator.com/item?id=38525954
Re: The worst thing about Jenkins is that it works (2019)
#264Earlier quoted context omitted.
Can't you use build time variables for this? https://docs.gitlab.com/ee/ci/variables/#pass-an-environment...
I tried this but it didn't work.
Re: The worst thing about Jenkins is that it works (2019)
#265Earlier quoted context omitted.
Random example: I want to define a setup job to build & push a base Docker image to our container registry. I then want to use this image as IMAGE in all subsequent jobs. This is impossible because the IMAGE field in YAML cannot be dynamic (determined at runtime) but I'd like to version/tag my image using the $CI_COMMIT_SHA.
Did you know that with Gitlab you can generate gitlab ci yaml in a job runtime and then run that yaml as a child pipeline using trigger:include:artifact? This was the only way I could create dynamic terraform pipelines which changed depending on a plan output. I'm sure could use it to achieve what you've described.
Re: The worst thing about Jenkins is that it works (2019)
#266Earlier quoted context omitted.
This is fine if you treat your CI provider as a "dumb shell runner". But good CI platforms have actually useful features and APIs (e.g. caching) and if you want to use them, a simple Makefile isn't going to work. For projects where the difference between a cold and warm cache build is tens of minutes, those features have meaningful quality of life improvements. This may be a tradeoff you're ok with, but for a lot of…
C++ with templating can take a while to compile, so for caching my build system would include mounting a Docker volume with contents of ~/.ccache/. This was well integrated with the rest of the scripts, so it would work equally fine if I ran them in my laptop or in the CI runner. My point is that I really don't believe that CI systems provide anything so unique that it couldn't be also provided by local software in a…
So I do agree with this. But I think there's nuance here.
The job of producing a build artifact involves steps that broadly break down into two categories: setting up the context, and doing the build. I totally agree that the "doing the build" bit should be a series of simple steps that are agnostic of their environment. A shell script, or a Makefile - something you can just invoke anywhere.
But the context bit is also super-important. On my laptop, I've already got the (e.g.) right JDK installed so that when I run `make`, the build succeeds. But I'm also not wiping my laptop before every build. On a CI platform, you're effectively starting from scratch every time, so whilst you can go and write your own code to set up the caching, download the tooling etc. etc. there's an enormous amount of value to be gained by re-using the CI-platform's software and features to do that as easily as possible. No-one wants to be writing code that works out how to go set up the right JDK in the right place, when `actions/setup-java` will basically do all that for you.
In theory, yes, you could go and curate a container image that has everything you need and just run your build inside that, but now you've got two bits of software to manage.
If you can't run the build locally, then yes, you're in a pickle.
Re: The worst thing about Jenkins is that it works (2019)
#267Re: The worst thing about Jenkins is that it works (2019)
#268Earlier quoted context omitted.
C++ with templating can take a while to compile, so for caching my build system would include mounting a Docker volume with contents of ~/.ccache/. This was well integrated with the rest of the scripts, so it would work equally fine if I ran them in my laptop or in the CI runner. My point is that I really don't believe that CI systems provide anything so unique that it couldn't be also provided by local software in a…
> My point is that I really don't believe that CI systems provide anything so unique that it couldn't be also provided by local software in a developer's laptop. If the question of "how do I build this in my laptop" is "you cannot, must use CI because we truly require some of its features", I'd consider that an ops failure. So I do agree with this. But I think there's nuance here. The job of producing a build artifac…
The closest I came was a GitHub actions job that did preflight checks to make sure all the right dependencies and sdks were present before calling into the same build script we use locally.
Don’t care where or how you set up your build environment, if the pfc passes your build should too. We still have some holdouts that do environment setup with a make file :/
Re: The worst thing about Jenkins is that it works (2019)
#269Earlier quoted context omitted.
> Phone os UI peak was iOS6. I couldn't disagree more strongly with this. The UI can't just be considered on its own without a discussion of the underlying functionality that it presents. iOS6 is like a caveman's OS compared to what today's OS versions can accomplish. So it may be true that iOS felt simple and intuitive, but it was also presenting so much less functionality that it really makes broad comparisons seem…
Most of the important functionality that’s been added didn’t require the changes to UI that I consider a downgrade.
For example, iMessage didn’t used to have any app integration to insert things. It used to be pictures/videos and text. Now it has the ability to insert a whole bunch of things and from a variety of first and third party apps. It can handle things like payments and location sharing. There has to be some kind of UI to handle that, and arguably there’s no way it can be “as clean” as an older version of the OS that simply didn’t have that functionality.
Another example: AirPlay 2 allows you to cast to multiple speakers at the same time and adjust volumes individually. You can also send audio from one app to one speaker and a different app to a different speaker and still play audio on the phone itself. So, now the AirPlay interface has radio buttons and more volume sliders, and it has a way to change which device’s audio you are controlling, and it has to fit and make sense somehow.
When the iPhone started there was just one volume bar for everything, so of course that UI was more intuitive - but it was also far less capable.
Re: The worst thing about Jenkins is that it works (2019)
#270Earlier quoted context omitted.
Confusing web design and desktop UI design in the 90s does little to dispute the assertion that desktop UI design peaked in the 90s. In the late 90s, the browser was completely new - so there was a lot of skeuomorphism, borrowing from other media to try to make the web work well. Space Jam's format was familiar, though to users of multimedia CD-ROM and other interactive hypermedia of the era. The desktop application…
I'm not confusing web and desktop UI design. I presented examples of both areas that were miseralbe in the 90's. There was not consistency between applications back then, only if you cherry-pick the ones that you like. Every Java GUI application would have a completely different UI from the base OS. Programs like America Online, RealPlayer, WinAmp, and Microsoft Bob, and Windows Media Player 7 and above would complet…
As did you. Every item you've cherry-picked used a multimedia app paradigm (similar to a CD ROM of the era) or was a novelty (Winamp). If you look a best-selling titles of the era, you'll find a lot of buy-in to both Mac and Windows HIG. Back then people bought word processors, spreadsheets, and other software and had a pretty high expectation for interop and usability compared to now.
> Did Microsoft Word run entirely in a web browser like it does today?
Of course not! Developers were still figuring out what you could do with primitive browsers and limited servers. Most developers from the 90s were unaware of the web until 97-98.