Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Lost Art of System Administration

matt-rickard.com

Post reply on HN
Print view View original

91–98 of 98 posts

Re: The Lost Art of System Administration

#91

Post by mbrodersen »

keyle wrote: post #8

I grew up with spectrums and ms-dos with a turbo and a reset button. I feel sad for my kids who will grow up with iPads and Windows 14 PCs with not the slightest clue on how things are running and the OS shielding them from crashing the thing or running viruses for fun. Same things for young lads that come out of university today with their degrees. They're well spoken and can recite algo theory like nothing but stil…

I am 100% sure we can’t reboot civilisation. All the easily accessible resources needed to bootstrap has been depleted years ago.

Re: The Lost Art of System Administration

#92

Post by orev »

mbrodersen wrote: post #90
throwaway892238 wrote: post #35

Most of those things listed are just technical details that change over time. Before all the Linux trivia there was the Solaris trivia, AIX trivia, Novell, HP/UX, Windows NT, OS/2, VMS, AS/400. Alpha, Sparc, VAX, tape robots, IPX/SPX, Token ring, VT100 dumb terminals. Old heads have forgotten more technical crap than exists today. (I can barely remember how to set up a Cisco router, and I used to work there.. I do, h…

I am a developer. Give me a VM I have root access to and walk away. That’s all I need from you.

Right. And then we’ll see how secure your app is running after you follow a bunch of random blog posts telling you to make all files world-writable, or to disable SELinux, or just turn off the firewall. And then the inevitable “it works for me” when we try to push to production and it all breaks.

Re: The Lost Art of System Administration

#93

Post by tasuki »

amtamt wrote: post #80
tasuki wrote: post #76

Earlier quoted context omitted.

You're of course right. But also, are you shipping all the logs ? Of course you're shipping application logs somewhere searchable, what about gc logs, syslog, kernel logs, etc? Is it never necessary to run ad hoc commands on your production systems?

How are gc/ syslog/ kernel logs are special and not sent to central logging? Ad-hoc logs (such as thread dumps etc.) can also be triggered via safe application end points from restricted locations. > "Is it never necessary to run ad hoc commands on your production systems?" Ad-hoc commands in production AS ROOT should be exception, not norm, and reserved for on-call members, not every developer working on team. Even…

Well done you, I guess? Nothing to disagree with. I'm just amazed this can be actual reality :)

Re: The Lost Art of System Administration

#94

Post by tasuki »

throwaway892238 wrote: post #89
tasuki wrote: post #62

Earlier quoted context omitted.

> No, I'm not giving you root on all the servers, tell me what you want to do. You just want to copy a file? You don't need root for that. I'm a developer, so I'm very biased. My interactions with system administrators were more along the lines of "I can't give you access to this system, you have to [come to my desk / hop on a zoom call] and tell me what to type into the shell." and me "Please type tail dash f, no, n…

Admittedly, that experience sucks. But there's a couple reasons they did that: 1. They could give you root access. But then you might start making other changes, and over time that makes everything unmaintainable. By them just typing in what you want this one time, they limit the scope of your access to just one change, they are aware of what changed, and can push back on it. It's basically a crappy change management…

1. I don't want root on prod servers, sounds like a nightmare!

2. Yes, I'm sure this was the case.

3. Indeed!

4. I haven't! My original comment just took some situation out of context. I know nothing about what else the sysadmins were doing, they probably had other things on their plate about which I knew nothing...

Re: The Lost Art of System Administration

#95

Post by mbrodersen »

orev wrote: post #92
mbrodersen wrote: post #90

Earlier quoted context omitted.

I am a developer. Give me a VM I have root access to and walk away. That’s all I need from you.

Right. And then we’ll see how secure your app is running after you follow a bunch of random blog posts telling you to make all files world-writable, or to disable SELinux, or just turn off the firewall. And then the inevitable “it works for me” when we try to push to production and it all breaks.

Just assume for a second that you are talking to an experienced competent developer here. I have never done any of the things you mention. And I have never worked with a developer who did any of the things you mention.

I have however worked with a narcissistic power hungry incompetent sys admin in the past. That doesn’t mean though that I automatically assume that all sys admins are the same. The one I am working with right now is highly competent and a pleasure to work with. And he is smart enough to simply give VM’s with root access to developers who ask for it.

Re: The Lost Art of System Administration

#96

Post by glotzerhotze »

tasuki wrote: post #76
amtamt wrote: post #70

Earlier quoted context omitted.

In a properly designed and deployed systems, no one (including sysadmin) should have a need to run tail/ less/ grep or whatever.... Logs should be in what ever centralized logging system is in use.. Metrics should be in what ever centralized metrics system is in use.. Config and secrets update should be possible from CI/ CD/ build system, without a full compile + test + deploy pipeline. Need of anyone having access t…

You're of course right. But also, are you shipping all the logs ? Of course you're shipping application logs somewhere searchable, what about gc logs, syslog, kernel logs, etc? Is it never necessary to run ad hoc commands on your production systems?

In an ideal world, you wouldn‘t need that. But most systems never matured to be considered being in an ideal state / world.

Re: The Lost Art of System Administration

#97

Post by glotzerhotze »

tasuki wrote: post #93
amtamt wrote: post #80

Earlier quoted context omitted.

How are gc/ syslog/ kernel logs are special and not sent to central logging? Ad-hoc logs (such as thread dumps etc.) can also be triggered via safe application end points from restricted locations. > "Is it never necessary to run ad hoc commands on your production systems?" Ad-hoc commands in production AS ROOT should be exception, not norm, and reserved for on-call members, not every developer working on team. Even…

Well done you, I guess? Nothing to disagree with. I'm just amazed this can be actual reality :)

It takes a while to get there - and a lot of discipline of those involved.

Re: The Lost Art of System Administration

#98

Post by gladiatr72 »

LMYahooTFY wrote: post #30
hsn915 wrote: post #25

The fact that linux systems need adminstration is one of the reasons why they suck. Imagine after buying a macbook that you also need to hire a sysadmin to keep it running smoothly? The idea is laughable. > Learning how to effectively use the command line. Unix philosophy. Pipes, scripting, and tooling. So much of programming is stitching things together. This is really bad. Much of the complexity in contemporary sof…

>The fact that linux systems need adminstration is one of the reasons why they suck. Do you buy and string together several MacBooks to run your business's custom built ERP system 24/7/365? There isn't a vast ecosystem of open source solutions to enterprise problems because that's not what macOS is, or attempts to be. I can't tell if this is a serious response or not, this comparison seems pretty ridiculous.

I realize you're being serious-but-funny, but I actually had to have that conversation with a development lead at a previous employer.
Post reply on HN