Software Developers Should Have Sysadmin Experience • Hacker News Board

Re: Software Developers Should Have Sysadmin Experience

#331

Post by jasonlotito » Mon, Jan 09, 2017, 6:59 PM UTC

mancerayder wrote: post #323

AlexB138 wrote: post #200

Earlier quoted context omitted.

As a Sys Admin turned Automation/Tools Engineer, I think you're missing part of the point. You've got the beginning of it right in saying that Sys Admins used to be involved in pinning down versions, and even in why that was necessary, but I believe you're incorrect in saying that the containerization technologies are bad for removing that. Those technologies don't exist so Developers can get around Sys Admins and ig…

Those technologies don't exist so Developers can get around Sys Admins and ignore your helpful advice. They exist to solve that problem that makes the Sys Admins role there necessary. It removes the underlying need for a Sys Admin to worry about the versions. Admins should see this as a good thing, but in my experience many dislike it because it takes them out of their Gatekeeper role. We shouldn't WANT to stop the D…

> However, we "Gatekeepers" are the ones that get paged and / or yelled at by a CTO when an application keels over. Not the developers.

And that's why people are moving away from that model. It's part of the reason DevOps is being embraced as a model. Developers should be on call to support the applications they build. You get benefits all around.

Re: Software Developers Should Have Sysadmin Experience

#332

Post by jasonlotito » Mon, Jan 09, 2017, 7:03 PM UTC

bandrami wrote: post #188

brianwawok wrote: post #184

Earlier quoted context omitted.

That's terrible. If I'm a Python dev, why should a sysadmin who doesn't even know Python tell me what version of a library I will use? I have regression tests to catch if an upgrade breaks anyyhing. What does a sysadmin have to approve or deny an upgrade? A little beard stroking and changelog reading? I think the movement towards containers is like you said, to keep sysadmins off the code. Sysadmins add value in sett…

why should a sysadmin who doesn't even know Python tell me what version of a library I will use Because he maintains that installation and you don't? But, yeah, that's why virtualenv, Docker, etc. were invented, because devs kept getting sick of installations having consequences. What does a sysadmin have to approve or deny an upgrade? Check for conflicts of this version of this library with other software currently…

> Because he maintains that installation and you don't?

Or not. You're devs are on-call, aren't they? They are maintaining their own software, right?

Re: Software Developers Should Have Sysadmin Experience

#333

Post by jdbernard » Mon, Jan 09, 2017, 7:05 PM UTC

mancerayder wrote: post #323

AlexB138 wrote: post #200

Earlier quoted context omitted.

As a Sys Admin turned Automation/Tools Engineer, I think you're missing part of the point. You've got the beginning of it right in saying that Sys Admins used to be involved in pinning down versions, and even in why that was necessary, but I believe you're incorrect in saying that the containerization technologies are bad for removing that. Those technologies don't exist so Developers can get around Sys Admins and ig…

Those technologies don't exist so Developers can get around Sys Admins and ignore your helpful advice. They exist to solve that problem that makes the Sys Admins role there necessary. It removes the underlying need for a Sys Admin to worry about the versions. Admins should see this as a good thing, but in my experience many dislike it because it takes them out of their Gatekeeper role. We shouldn't WANT to stop the D…

The problem with your characterization of things is companies like the one I'm in where us "Developers" have replaced the gatekeepers entirely. We have no dedicated SysOps, and yet our production environment stays up just fine.

Re: Software Developers Should Have Sysadmin Experience

#334

Post by snuxoll » Mon, Jan 09, 2017, 7:09 PM UTC

mancerayder wrote: post #323

AlexB138 wrote: post #200

Earlier quoted context omitted.

As a Sys Admin turned Automation/Tools Engineer, I think you're missing part of the point. You've got the beginning of it right in saying that Sys Admins used to be involved in pinning down versions, and even in why that was necessary, but I believe you're incorrect in saying that the containerization technologies are bad for removing that. Those technologies don't exist so Developers can get around Sys Admins and ig…

Those technologies don't exist so Developers can get around Sys Admins and ignore your helpful advice. They exist to solve that problem that makes the Sys Admins role there necessary. It removes the underlying need for a Sys Admin to worry about the versions. Admins should see this as a good thing, but in my experience many dislike it because it takes them out of their Gatekeeper role. We shouldn't WANT to stop the D…

Sounds like an organizational failure to me.

When I started at my current company the traditional silo between dev and systems was there (although we were allowed to deploy our own stuff) - they managed everything we ran our apps on and we just deployed them to servers they had already configured. Over the past ~3 years we've made a lot of changes, the department manager for our IS team is present in our daily standup calls to relay information between our two teams and we now have a couple separate VMWare clusters dedicated to our applications and VM running on them is our responsibility for the most part. We are the first to get called for issues with our applications, and where necessary we work collaboratively with our systems team to resolve them - we don't throw blame around, it does no good.

I should add most of this is only possible because we have real DevOps people on our team (well, really, it's just me right now - we lost our other and need to hire a replacement still) - not developers who know enough to copy a blob of crap to a server to run, but people who have real skills in both aspects. We are trusted to maintain things because we can do it right, and while it took a lot of work (and some unfortunate infighting) to get to this point both of our departments are working great with this arrangement.

There's still kinks that need ironing, we've not done an adequate job at writing documentation so our systems team can help with some failures (primarily on our Linux VM's, our whole systems team is Windows admins) if we aren't available - but it's on the radar as well as getting PagerDuty set up to escalate alerts to them if we don't respond in time (like having our PostgreSQL data volume fill up over the weekend, not a call I wanted to get at 10AM on Sunday).

So yeah, fix your culture issue, get people communicating daily between your teams, share responsibility for issues instead of placing blame.

Re: Software Developers Should Have Sysadmin Experience

#335

Post by [deleted] » Mon, Jan 09, 2017, 7:12 PM UTC

crdoconnor wrote: post #304

jstimpfle wrote: post #211

Earlier quoted context omitted.

Another perspective: Why don't you want your software to be compatible with the system that your sysadmin provides? (Assuming that system is not completely obsolete). Minimize your dependencies. It's incidentally also what leads to clean code bases.

>Minimize your dependencies. It's incidentally also what leads to clean code bases. Oh hell no. I have wasted far too much of my life maintaining buggy, technical debt ridden reinvented wheels where there was a well maintained package that could just have been used instead.

[deleted]

Re: Software Developers Should Have Sysadmin Experience

#336

Post by [deleted] » Mon, Jan 09, 2017, 7:16 PM UTC

crdoconnor wrote: post #304

jstimpfle wrote: post #211

Earlier quoted context omitted.

Another perspective: Why don't you want your software to be compatible with the system that your sysadmin provides? (Assuming that system is not completely obsolete). Minimize your dependencies. It's incidentally also what leads to clean code bases.

>Minimize your dependencies. It's incidentally also what leads to clean code bases. Oh hell no. I have wasted far too much of my life maintaining buggy, technical debt ridden reinvented wheels where there was a well maintained package that could just have been used instead.

[deleted]

Re: Software Developers Should Have Sysadmin Experience

#337

Post by jdbernard » Mon, Jan 09, 2017, 7:19 PM UTC

gaius wrote: post #292

jstimpfle wrote: post #273

Earlier quoted context omitted.

> To respond directly to your question: because there are new things available in new libraries that allow us to develop new features! If it were only that, we would have an easy time. The new things you need to develop new features are far and far between.

The new things you need to develop new features are far and far between 99% of web software written these days could fulfil identical use cases on an IBM 3270 from 40 years ago. You enter something into a form and it gets stored in a database. You enter something into a field and it generates a report. That's all Amazon, Facebook, Google, any e-commerce site are. Sure it might be nice to use a new version of that new…

And yet none of those things did arise 40 years ago. All of the nuances of all the code written since then make a difference, despite duplicating "identical use cases".

Re: Software Developers Should Have Sysadmin Experience

#338

Post by serge2k » Mon, Jan 09, 2017, 7:24 PM UTC

mancerayder wrote: post #323

AlexB138 wrote: post #200

Earlier quoted context omitted.

As a Sys Admin turned Automation/Tools Engineer, I think you're missing part of the point. You've got the beginning of it right in saying that Sys Admins used to be involved in pinning down versions, and even in why that was necessary, but I believe you're incorrect in saying that the containerization technologies are bad for removing that. Those technologies don't exist so Developers can get around Sys Admins and ig…

Those technologies don't exist so Developers can get around Sys Admins and ignore your helpful advice. They exist to solve that problem that makes the Sys Admins role there necessary. It removes the underlying need for a Sys Admin to worry about the versions. Admins should see this as a good thing, but in my experience many dislike it because it takes them out of their Gatekeeper role. We shouldn't WANT to stop the D…

and we get yelled at when we can't deliver a feature because some gatekeeper is sitting atop his little throne in the kingdom of servers saying no. ;)

This is institutional failure.

Re: Software Developers Should Have Sysadmin Experience

#339

Post by jstimpfle » Mon, Jan 09, 2017, 7:29 PM UTC

serge2k wrote: post #338

mancerayder wrote: post #323

Earlier quoted context omitted.

Those technologies don't exist so Developers can get around Sys Admins and ignore your helpful advice. They exist to solve that problem that makes the Sys Admins role there necessary. It removes the underlying need for a Sys Admin to worry about the versions. Admins should see this as a good thing, but in my experience many dislike it because it takes them out of their Gatekeeper role. We shouldn't WANT to stop the D…

and we get yelled at when we can't deliver a feature because some gatekeeper is sitting atop his little throne in the kingdom of servers saying no. ;) This is institutional failure.

Example?

Re: Software Developers Should Have Sysadmin Experience

#340

Post by serge2k » Mon, Jan 09, 2017, 7:31 PM UTC

brightball wrote: post #270

treehau5 wrote: post #246

Earlier quoted context omitted.

My comment to you two is -- why not both? Stability, sanity, all that is amazing, and a must have. But also bug-fixes, security improvements, and performance improvements are wonderful too, which tends to come with using up-to-date dependencies. The problem with the latter, as you mentioned, is when it introduces breaking API changes and is wholly not backwards compatible. This is not a "kids playing with toys wantin…

Backwards compatibility is seriously under appreciated. When I tell developers to ensure that their changes are backwards compatible, they tend to look at me like I'm green. I do not understand the disconnect that developers have with understanding all of the benefits that it brings. Yes, you have some extra code in your code base so it's less clean. You also have a stable environment as a result. The first affects o…

> Yes, you have some extra code in your code base so it's less clean. You also have a stable environment as a result. The first affects only your personal preference. The latter affects all of your developers and users.

No, the former results in a bloated code base full of old legacy crap that no one understands and is afraid to touch because it might break. You have to insert weird workarounds because that bug is now a feature to some idiot and you provide backwards compatibility so it lives forever now.