Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Advent of Sysadmin 2025

sadservers.com

Post reply on HN
Print view View original

101–110 of 140 posts

Re: Advent of Sysadmin 2025

#101

Post by maratc »

n4bz0r wrote: post #56
bionsystem wrote: post #48

Earlier quoted context omitted.

Yeah I was thinking of using it for us actually. Connects to everything, lots of plugins, etc. I wonder what the hate is from, they are all pretty bad aren't they ? Will test forgejo's CI first as we'll use the repo anyway, but if it ain't for me, it's going to be jenkins I assume.

Cons: - DSL is harder to get into. - Hard to reproduce a setup unless builds are in DSL and Jenkins itself is in a fixed version container with everything stored in easily transferable bind volumes; config export/import isn't straightforward. - Builds tend to break in a really weird way when something (even external things like Gitea) updates. - I've had my setup broken once after updating Jenkins and not being able…

CASC plugin + seed jobs keep all your jobs/configurations in files and update them as needed, and k8s + Helm charts can keep the rest of config (plugins, script approvals, nodes, ...) in a manageable file-based state as well.

We have our main node in a state that we can move it anywhere in a couple of minutes with almost no downtime.

I'll add another point to "Pros": Jenkins is FOSS and it costs $0 per developer per month.

Re: Advent of Sysadmin 2025

#102

Post by stackskipton »

technion wrote: post #46
0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

I know its a common view that sysadmin/devops are the same these days, but witha current sysadmin role nothing youve mentioned sounds relevant. Let's give you my list: 1. Patch Microsoft exchange with only a three hour outage window 2. Train a user to use onedrive instead of emailing 50mb files and back and forth 3. Setup eight printers for six users. Deal with 9gb printer drivers. 4. Ask an exec if he would please l…

Former Exchange Admin here: 1 is easy, I used to do 70k mailboxes in middle of the day only but it requires spare hardware or virtualization with headroom.

Deploy new Server(s), patch, install Exchange, Setup DAGs, migrate everyone mailbox, swing load balancer over to new servers, uninstall Exchange from old, remove old from Active Directory, delete servers.

BTW, Upgrades now suck because Office365 uses method above so upgrade system never gets good Q&A from them.

Re: Advent of Sysadmin 2025

#103

Post by fduran »

kralos wrote: post #66
fduran wrote: post #23

Earlier quoted context omitted.

hello, creator here, sorry about that. In this case you can click again on the "Open the Server Terminal in a New Window" button

It would be cool if we could SSH into the temporary host (I'm guessing these hosts currently aren't internet connected to avoid abuse so might not be possible or require some super careful firewalling)

Hello, SadServers guy here. Free VMs are sandboxed (no way in or out other than coming in through the proxy) for security reasons. Paid accounts have VMs with internet access and SSH access (and your pub key is added to all VMs for convenience)

Re: Advent of Sysadmin 2025

#106

Post by fduran »

udev4096 wrote: post #38

I wonder if we could get something like that for k8s, docker and other container ecosystem

Hello, SadServers guy here.

We have scenarios running on k8s, both on single VMs (the ones you can see in the scenario list) and we also have a beta/PoC k8s cluster where we currently run a couple of scenarios as single pod (a docker container) or as a full system (the "kubernetes playgrounds", which is kind of hidden while we test it).

Is this what you were wondering? we do have pending to introduce podman scenarios as well

Re: Advent of Sysadmin 2025

#107

Post by EvanAnderson »

stackskipton wrote: post #102
technion wrote: post #46

Earlier quoted context omitted.

I know its a common view that sysadmin/devops are the same these days, but witha current sysadmin role nothing youve mentioned sounds relevant. Let's give you my list: 1. Patch Microsoft exchange with only a three hour outage window 2. Train a user to use onedrive instead of emailing 50mb files and back and forth 3. Setup eight printers for six users. Deal with 9gb printer drivers. 4. Ask an exec if he would please l…

Former Exchange Admin here: 1 is easy, I used to do 70k mailboxes in middle of the day only but it requires spare hardware or virtualization with headroom. Deploy new Server(s), patch, install Exchange, Setup DAGs, migrate everyone mailbox, swing load balancer over to new servers, uninstall Exchange from old, remove old from Active Directory, delete servers. BTW, Upgrades now suck because Office365 uses method above…

Same feeling here re: migrations being easy if the Customer isn't a cheapass. Small business Customers who had the competing requirements of spending as little money as possible and having as much uptime as possible were the stressor.

Re: Advent of Sysadmin 2025

#108

Post by fduran »

dontdoxxme wrote: post #61

Without sharing too many spoilers... I solved the challenge but the check script was unhappy. The curl commands in the script worked fine, the earlier parts of the script failed, i.e. it didn't like how I'd decided to make that work. This kind of thing annoys me. This is why CTFs are great, where the goal is to get the flag string. Obviously harder to do for sysadmin, but expecting a particular configuration when I m…

hello, thanks for the feedback. Just deployed a new image that only checks for the objective, not at what docker network somebody uses.

It is hard to have a checker that eliminates both false positives and false negatives in general, but we always try to minimize false negatives and we failed initially here.

Re: Advent of Sysadmin 2025

#109

Post by maratc »

0xbadcafebee wrote: post #98
jagged-chisel wrote: post #19

Earlier quoted context omitted.

> … from Jenkins to GitHub Actions. Oh, good lord why?

Many, many reasons... the most important of which is, Jenkins is a constant security nightmare and a maintenance headache. But also it's much harder to manage a bunch of random Jenkins servers than GHA. Authentication, authorization, access control, configuration, job execution, networking, etc. Then there's the configuration of things like env vars and secrets, environments, etc that can also scale better. I agree G…

I really scratch my head when I read your comment, as nothing of this is a real issue in my Jenkins.

> bunch of random Jenkins servers

Either PXE boot from an image, or k8s from an image, have a machine or pod rebooted/destroyed after one job. Update your image once a month, or have a Jenkins job to do that for you.

> Authentication, authorization, access control

Either use LDAP or Login via Github, and Matrix security plugin. Put all "Devops" group into admins, the rest into users, never touch it again.

> configuration

CASC plugin and seed for jobs, and/or Helm for just about everything else.

> env vars and secrets

Pull everything from Vault with Vault plugin.

> as a sysadmin Jenkins will suck the life out of you

I spend about 1-2 hours a week managing Jenkins itself, and the rest of the week watching the jobs or developing new ones.

Re: Advent of Sysadmin 2025

#110

Post by Waterluvian »

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

Here’s the first step to all of these that I often see sysadmins stumbling on: communicate in written, non-abstract terms why each of these matter.

Most are obvious to most people. None are obvious to everybody.

Post reply on HN