EDIT: Ah, ok, `vi` is installed on the server _itself_, just not in the Docker containers. So I guess I'm going to have to `docker cp` them in. Can do o7
Advent of Sysadmin 2025
131–140 of 140 posts
Re: Advent of Sysadmin 2025
#132Earlier quoted context omitted.
I really scratch my head when I read your comment, as nothing of this is a real issue in my Jenkins. > bunch of random Jenkins servers Either PXE boot from an image, or k8s from an image, have a machine or pod rebooted/destroyed after one job. Update your image once a month, or have a Jenkins job to do that for you. > Authentication, authorization, access control Either use LDAP or Login via Github, and Matrix securi…
Well one issue is, CasC isn't enough. You often have to write JobDSL to get around some limitation in CasC, and sometimes Groovy for limitations in the other two. If you want to manage access control (and you choose the correct Auth plugin, and figure out how to configure it), often you need an admin to make changes in both the Jenkins server and your backend AuthNZ system. Then there's the "seed job vs not-seed-job"…
But yeah, if all there is to do is "git clone && build && deploy" then Jenkins is an overkill and it probably wasn't warranted in the first place.
Re: Advent of Sysadmin 2025
#133Cool, might try it out! Are there any solutions repositories for them. I’d love to get an explanation for the ones I’m about to fail.
Re: Advent of Sysadmin 2025
#134Earlier quoted context omitted.
Well one issue is, CasC isn't enough. You often have to write JobDSL to get around some limitation in CasC, and sometimes Groovy for limitations in the other two. If you want to manage access control (and you choose the correct Auth plugin, and figure out how to configure it), often you need an admin to make changes in both the Jenkins server and your backend AuthNZ system. Then there's the "seed job vs not-seed-job"…
My issue with GHA and other "dead simple" systems is that my CI is complicated. Having a real programming language for stuff like "calculate what date it was a week ago" or "concatenate these three strings but only under some conditions" or "parse the output and build an object out of it" is really helpful while a bastardised YAML-based Jinja template simply can't hold up. But yeah, if all there is to do is "git clon…
Re: Advent of Sysadmin 2025
#135Earlier quoted context omitted.
Not in love with its insistence on recreating the container from scratch every step of the pipeline, among a bundle of other irksome quirks. There are certainly worse choices, though.
You don't actually have to. If you use the built in container registry and build artifacts, you can pass between steps.
Re: Advent of Sysadmin 2025
#136Earlier quoted context omitted.
This is documented currently (supposed to be 14 days). [1] That said, I have found runners to be unnecessarily difficult. But Jenkins and its own quirks, and when I used GitLab, it used ancient docker-machine and outdated AMIs by default. I think Buildkite has been the only one to make this easy and scalable. But it is meant for self hosted runners. [1] https://docs.github.com/en/enterprise-cloud@latest/actions/h...
Buildkite also has hosted runners (which they all agents): https://buildkite.com/docs/pipelines/hosted-agents
It was originally (and still usually) used by those who wanted to self-host runners.
Re: Advent of Sysadmin 2025
#137Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…
I know its a common view that sysadmin/devops are the same these days, but witha current sysadmin role nothing youve mentioned sounds relevant. Let's give you my list: 1. Patch Microsoft exchange with only a three hour outage window 2. Train a user to use onedrive instead of emailing 50mb files and back and forth 3. Setup eight printers for six users. Deal with 9gb printer drivers. 4. Ask an exec if he would please l…
Ask?! This is where the org's cyber insurance is your friend. Just have the executive get the provider's clearance on him not having MFA. I'm sure that line item will change his mind, and if not, be sure to accidently mention those exemptions to those yearly auditors.
Re: Advent of Sysadmin 2025
#138Earlier quoted context omitted.
My issue with GHA and other "dead simple" systems is that my CI is complicated. Having a real programming language for stuff like "calculate what date it was a week ago" or "concatenate these three strings but only under some conditions" or "parse the output and build an object out of it" is really helpful while a bastardised YAML-based Jinja template simply can't hold up. But yeah, if all there is to do is "git clon…
For complex logic I don't rely on the CI system; I've been burned too many times. I shell out to an external program and have it return an output variable, and I just do "if $foo = y then blah" in the CI's DSL (and I keep those tests to a minimum; rather have more separate jobs than one complex job). Often I will put everything in a dedicated build tool (Make or similar) so I can run it from my laptop or CI, and any…
I do that too, until my complex logic belongs to the CI system and nowhere else.
As an example:
It's 2 am now and Jenkins needs to run some (but not all) nightly tests[0]. To figure out which, let's bring the source code and analyse the configuration file, disregarding anything that doesn't need to run at this hour. Once we have the plan for which tests to run, let's figure out what we need to build[1]. Also, let's see what is the status of the pool running the tests[2] so we can decide on a tests parallelisation strategy for this run. When we have a plan, let's build and test, keeping an eye on the triggered tests. When all these have finished, analyse the failures[3], create tickets for the failures[4] and prepare a report to be sent.
I wouldn't be able to express all this in YAML.[0] other "nightly tests" run at 1, 3, 4 etc.
[1] this is mapped in the configuration file too.
[2] this is internal to Jenkins
[3] same
[4] this involves finding the "responsible person," so a lot of API calls
Re: Advent of Sysadmin 2025
#139Earlier quoted context omitted.
For complex logic I don't rely on the CI system; I've been burned too many times. I shell out to an external program and have it return an output variable, and I just do "if $foo = y then blah" in the CI's DSL (and I keep those tests to a minimum; rather have more separate jobs than one complex job). Often I will put everything in a dedicated build tool (Make or similar) so I can run it from my laptop or CI, and any…
> For complex logic I don't rely on the CI system I do that too, until my complex logic belongs to the CI system and nowhere else. As an example: It's 2 am now and Jenkins needs to run some (but not all) nightly tests[0]. To figure out which, let's bring the source code and analyse the configuration file, disregarding anything that doesn't need to run at this hour. Once we have the plan for which tests to run, let's…
Fwiw, I do believe you can do this in GHA, but you may need to call their API from your workflow. In addition, their replacement for Groovy is to run an action which lets you embed Javascript/Typescript and call their SDK. It sucks, but so does Groovy! ;-)
Re: Advent of Sysadmin 2025
#140[flagged]
Checking out how the platform works was two clicks away: home -> give me a server. I don't know of any other SaaS which gives you a VM with one click without any registration but we do it. In any case thanks for the feedback, I've put a button on this /advent page for clarity, cheers
> Sign up for a free account (needed to keep track of your progress)
is a complete lie. Tracking a person’s progress is what cookies are for. You don’t need us to create an account for that.
What you do need users to create accounts for, is for you to track every user and their progress.