Advent of Sysadmin 2025 • Hacker News Board

Re: Advent of Sysadmin 2025

#91

Post by rtp4me » Mon, Dec 01, 2025, 3:08 PM UTC

f1shy wrote: post #67

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

>> Sysadmin/DevOps (they're synonyms now!) Is this really like that? Isn't there any Unix/DBA anymore? I associate DevOps to what at my time we called "operations" and "development". We had 5 teams or so: 1) Developers, who would architect and write code, 2) Operations who would deploy, monitor and address customer complaints, 3) Unix (aka SYS) administrators, who would take care of housekeeping of well, the OS (and…

For 4) - consider PGHero[1] and PGTuner[2] instead of a full-time DBA. We use both in production and they work very well to help track down performance issues with Postgres.

[1] https://github.com/ankane/pghero

[2] https://pgtune.leopard.in.ua/

Edit: For the record, I have worked at a few small companies as the "SysAdmin" guy who did the whole compliment of servers, OS, storage, networking, VMs, DB, perf tuning, etc.

Re: Advent of Sysadmin 2025

#92

Post by jabroni_salad » Mon, Dec 01, 2025, 3:09 PM UTC

technion wrote: post #60

hansmayer wrote: post #51

Earlier quoted context omitted.

What you describe sounds more like a MS "Modern Workplace" / IT support in a corporate environment.

Are we arguing that corporate workers arent "real sysadmins"?

HN culture as a whole doesnt really recognize the validity of business that buy software vs build software.

Re: Advent of Sysadmin 2025

#93

Post by infogulch » Mon, Dec 01, 2025, 3:15 PM UTC

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

Q: 3. Get a user to upgrade their app's dependencies to versions newer than 2010.

A: Calculate the average age in years of all dependencies calculated by: (max(most recent version release date, date of most recent CVE on library) - used version release date). Sleep for that many seconds before the app starts.

Re: Advent of Sysadmin 2025

#94

Post by paulddraper » Mon, Dec 01, 2025, 3:23 PM UTC

cobertos wrote: post #28

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

Re: 6. ... Github Actions Github Actions left a bad taste in my mouth after having it randomly removed authenticated workers from the pool, after their offline for ~5 days. This was after setting up a relatively complex PR workflow (always on cheap server starts up very expensive build server with specific hardware) only to have it break randomly after a PR didn't come in for a few days. And no indication that this h…

This is documented currently (supposed to be 14 days). [1]

That said, I have found runners to be unnecessarily difficult.

But Jenkins and its own quirks, and when I used GitLab, it used ancient docker-machine and outdated AMIs by default.

I think Buildkite has been the only one to make this easy and scalable. But it is meant for self hosted runners.

[1] https://docs.github.com/en/enterprise-cloud@latest/actions/h...

Re: Advent of Sysadmin 2025

#95

Post by DoctorOW » Mon, Dec 01, 2025, 3:33 PM UTC

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

> Get a user to use configuration management rather than scp'ing config files from their laptop to the server.

Damn, this one I'm guilty of. Though, I'm not real Sysadmin/DevOps, I'm just throwing something together and deploying it on a LAN-only VM for security reasons (I don't trust the type of code I would write)

Re: Advent of Sysadmin 2025

#96

Post by panzagl » Mon, Dec 01, 2025, 3:46 PM UTC

Nextgrid wrote: post #69

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

> Get a user to stop logging in as root. It really depends if the machine is hosting anything that you don't want some users to access. If the machine is single-purpose and any user is already able to access everything valuable from it (DB with customer data, etc) or trivially elevate to root (via sudo, docker access, etc) then it's just pointless extra typing and security theatre.

I guess no one ever audits your servers.

Re: Advent of Sysadmin 2025

#97

Post by john-carter » Mon, Dec 01, 2025, 3:52 PM UTC

[dead]

Re: Advent of Sysadmin 2025

#98

Post by 0xbadcafebee » Mon, Dec 01, 2025, 4:07 PM UTC

jagged-chisel wrote: post #19

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

> … from Jenkins to GitHub Actions. Oh, good lord why?

Many, many reasons... the most important of which is, Jenkins is a constant security nightmare and a maintenance headache. But also it's much harder to manage a bunch of random Jenkins servers than GHA. Authentication, authorization, access control, configuration, job execution, networking, etc. Then there's the configuration of things like env vars and secrets, environments, etc that can also scale better. I agree GHA kinda sucks as a user tool, but as a sysadmin Jenkins will suck the life out of you and sap your time and energy that can go towards more important [to the company] tasks.

Re: Advent of Sysadmin 2025

#99

Post by philipwhiuk » Mon, Dec 01, 2025, 4:18 PM UTC

alberth wrote: post #26

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

I’d be super interested to see solutions to each, just to learn from.

You can deploy tooling (e.g. BeyondTrust / CyberArk for 1&2), but ultimately there's a conversation and a migration plan to be done for each.

Re: Advent of Sysadmin 2025

#100

Post by 0xbadcafebee » Mon, Dec 01, 2025, 4:25 PM UTC

technion wrote: post #46

0xbadcafebee wrote: post #14

Here's 12 Sysadmin/DevOps (they're synonyms now!) challenges, straight from the day job: 1. Get a user to stop logging in as root. 2. Get all users to stop sharing the same login and password for all servers. 3. Get a user to upgrade their app's dependencies to versions newer than 2010. 4. Get a user to use configuration management rather than scp'ing config files from their laptop to the server. 5. Get a user to bak…

I know its a common view that sysadmin/devops are the same these days, but witha current sysadmin role nothing youve mentioned sounds relevant. Let's give you my list: 1. Patch Microsoft exchange with only a three hour outage window 2. Train a user to use onedrive instead of emailing 50mb files and back and forth 3. Setup eight printers for six users. Deal with 9gb printer drivers. 4. Ask an exec if he would please l…

I used to have that job, but my title wasn't Sysadmin, it was IT Manager. For companies small enough that they don't have multiple roles, you do both... but for larger companies, the user-side stuff is done by IT, and the server-side stuff is done by a Sysadmin. (And my condolences; having done that combined role, it's not easy, and you don't get paid enough!)