Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software Developers Should Have Sysadmin Experience

blog.professorbeekums.com

Post reply on HN
Print view View original

211–220 of 424 posts

Re: Software Developers Should Have Sysadmin Experience

#211

Post by jstimpfle »

brianwawok wrote: post #184
bandrami wrote: post #164

As a grumpy evil sysadmin, I think the good Professor misses where the real disconnect is, at least nowadays: stack management. Why do things like Docker exist? Because developers got tired of sysadmins saying "sorry, you can't upgrade Ruby in the middle of this project". Why does virtualenv exist? A similar reason. Containerized ecosystems (which is to say basically all of them now) are really a sign of those of us…

That's terrible. If I'm a Python dev, why should a sysadmin who doesn't even know Python tell me what version of a library I will use? I have regression tests to catch if an upgrade breaks anyyhing. What does a sysadmin have to approve or deny an upgrade? A little beard stroking and changelog reading? I think the movement towards containers is like you said, to keep sysadmins off the code. Sysadmins add value in sett…

Another perspective: Why don't you want your software to be compatible with the system that your sysadmin provides? (Assuming that system is not completely obsolete).

Minimize your dependencies. It's incidentally also what leads to clean code bases.

Re: Software Developers Should Have Sysadmin Experience

#212

Post by sbov »

0xC0DECAFE2020 wrote: post #51
Aloha wrote: post #39

Earlier quoted context omitted.

I've worked with all kinds - from windows devs who can't figure out how to install visual studio - to people who understand Windows, Linux, and macOS - as well as basic system administration for each platform. The people who are most successful at rapidly developing good high quality software are more in the later group. Would you trust a RF engineer who couldn't troubleshoot his own radio designs? why would you trou…

I recently questioned someone about this very subject. They wanted to hire a "CSS expert" because the "developer" didn't have a grasp of css after having developed the project in JS/HTML. I was so confused as to how that's possible.

I suck at CSS and do a lot of js/html, but most of my work is on the back end. I can do ok with CSS but it will definitely take me longer than someone who knows what they're doing. Usually we contract out the design/CSS for a few pages and I adopt that for the rest of the website.

Re: Software Developers Should Have Sysadmin Experience

#213

Post by devdas »

FLUX-YOU wrote: post #178
pjmorris wrote: post #12

Should sysadmins have software development experience (e.g. DevOps)? For what values should 'X have Y experience?' Should we go as far as... "A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new proble…

Development is sadly the job that is becoming the kitchen sink of "You ought to have skill X" where X is anything from sysadmin, computer science, business, security, product, bare-metal, networking, infrastructure, management, statistics, operating systems, math, social, and industry knowledge. If you aggregate all of the "Every developer should know X" posts and blogs, the list would probably be very long. It only…

Writing software is less useful than writing working, relevant software. The latter needs far more breadth than simply writing code.

Re: Software Developers Should Have Sysadmin Experience

#214

Post by jstimpfle »

cs02rm0 wrote: post #50

This might be controversial, but I don't think you get to be a half decent developer without being a reasonable sysadmin. Maybe my experience is unusual, but I've never worked anywhere that the sysadmins knew more than the developers about how best to run their code in production. And when things go wrong with it how best to find the cause of the issue. And I've never thrown code over a wall without having tested it…

> but I don't think you get to be a half decent developer without being a reasonable sysadmin

I think you've just described me. And I don't see the argument against sysadmin.

> leaving $800/day contract developers waiting for builds that run in 20 seconds on their laptops take nearly an hour.

Sorry I just can't take you seriously.

Re: Software Developers Should Have Sysadmin Experience

#215

Post by btreecat »

bandrami wrote: post #188
brianwawok wrote: post #184

Earlier quoted context omitted.

That's terrible. If I'm a Python dev, why should a sysadmin who doesn't even know Python tell me what version of a library I will use? I have regression tests to catch if an upgrade breaks anyyhing. What does a sysadmin have to approve or deny an upgrade? A little beard stroking and changelog reading? I think the movement towards containers is like you said, to keep sysadmins off the code. Sysadmins add value in sett…

why should a sysadmin who doesn't even know Python tell me what version of a library I will use Because he maintains that installation and you don't? But, yeah, that's why virtualenv, Docker, etc. were invented, because devs kept getting sick of installations having consequences. What does a sysadmin have to approve or deny an upgrade? Check for conflicts of this version of this library with other software currently…

>Because he maintains that installation and you don't?

So why is that an acceptable mentality for "in-house" developed software but if you buy something proprietary from a third party where you have zero say over what lib/langs are used, it's A-OK?

>Check for conflicts of this version of this library with other software currently in use (by other developers maybe, or even by the same developer).

That's not the case when using containers properly. Every service gets it's own environment so whatever version of lib-xyz is needed, even if incompatible with other parts of the project, are walled off for only the service that needs it.

>Add it to the watchlist on the dozen or so security mailing lists and newsfeeds he checks daily.

Ok this is where I completely agree with you as we have been working on this at our company. My personal solution seems pretty logical though so hear me out.

1) Build a docker file that fully documents the install of your service as well as any OS level dependencies. Ensure that any config files are external to the container to allow sysadmin access.

2) Document in a central location (say an internal wiki) what the external services, servers, repositories, developers, and admins are responsible for the service.

3) Automate builds of containers from repo and add automated testing post containerization.

4) Sysadmins monitor repositories for changes to docker files or wiki articles for new services, databases and libraries as well as taking note of library versions. If an issue with a particular lib or service is discovered, the config files can be edited to point to a new service. Or a new container build can be triggered with zero changes to the source code, but a forced update to the OS packages for the container.

In a tight situation where a developer might not be available on-call, the sysadmins have more control over a similar proprietary product but don't have a workflow for messing with source code (which they are likely not familiar with regardless).

There are solutions to the issues you raise (often trivial ones at that), they just require an adjustment to workflow and an increase in communication between developers and their sysadmins.

Re: Software Developers Should Have Sysadmin Experience

#216

Post by awinder »

bandrami wrote: post #164

As a grumpy evil sysadmin, I think the good Professor misses where the real disconnect is, at least nowadays: stack management. Why do things like Docker exist? Because developers got tired of sysadmins saying "sorry, you can't upgrade Ruby in the middle of this project". Why does virtualenv exist? A similar reason. Containerized ecosystems (which is to say basically all of them now) are really a sign of those of us…

  > 15 years ago, when a project would kick-off, as a sysadmin 
  > I'd be invited in and the developers and I would hash out
  > what versions of each language and library involved the
  > project would use
To wit: the new ideologies in infrastructure management are actually designed to solve the underlying problem that necessitated that kind of working setup. Why should the version of a lib in one part of the software somehow pose existential threat to the infrastructure? Engrain the dependencies into contained, independently deployable pieces, and make it so that app-level code can evolve without bringing down the world with it. Make it easy to revert back, and/or utilize phased rollouts, and you've got the ability to iterate quickly, keep pace with external dependencies, and it no longer has to be some scary thing that requires big back-and-forth meetings over mundane details.

(As for software that releases often, maybe it's an over-correction, but there's a reason things don't work as they did in the glory days, and that's because they were never really that glorious.)

This doesn't necessarily rule out the expertise of systems administration, because the platforms for all of this need to be built & maintained, and there's still a lot of work to be done on network boarder security, etc. It's a movement that focuses systems administration to systems administration, instead of having to be this big org arbiter of microdecisions, and all the baggage that goes along with trying to be the gatekeeper of all.

Re: Software Developers Should Have Sysadmin Experience

#217

Post by grzm »

kefka wrote: post #199
krylon wrote: post #105

Earlier quoted context omitted.

AMEN!!! In my current job, I am wearing both hats, and while I like that there is a certain variety in my work, users calling for help is highly disruptive when programming or doing some other stuff that requires deep focus. The upside that in a three-person IT department there is very little bureaucracy to fight, just the odd "organically grown" legacy system.

As an aside, did you know that the word "Amen" actually is a acronym in the Jewish language that means "El melekh neʾeman" (or AMN) which translates to "God, trustworthy King". (source: https://en.wikipedia.org/wiki/Amen ) I figured the etymology of that word was rather interesting. But yeah, I get the whole SysAd/Dev dual job. They're tough to balance and do effectively. SysAds are firefighters. When the nag(ios) al…

From that Wikipedia same article, it says

The Talmud teaches homiletically that the word amen is an acronym

The etymology section shows the word has much more prosaic roots. The Talmudic acronym seems to rather be an interesting backronym.

Re: Software Developers Should Have Sysadmin Experience

#219

Post by lmm »

eternalban wrote: post #165
lmm wrote: post #148

Earlier quoted context omitted.

There's no value in arguing over the semantics of "engineering". There are huge differences between software development and e.g. civil engineering, to the point that I would be dubious about any analogy that treated them as the same thing.

Chemical engineering, Hydroelectrical engineering, Power engineering come immediately to mind as engineering disciplines that deal with active systems that require operational management and control.

Sure, but those are still all very different from software development.

Re: Software Developers Should Have Sysadmin Experience

#220

Post by jstimpfle »

awinder wrote: post #216
bandrami wrote: post #164

As a grumpy evil sysadmin, I think the good Professor misses where the real disconnect is, at least nowadays: stack management. Why do things like Docker exist? Because developers got tired of sysadmins saying "sorry, you can't upgrade Ruby in the middle of this project". Why does virtualenv exist? A similar reason. Containerized ecosystems (which is to say basically all of them now) are really a sign of those of us…

> 15 years ago, when a project would kick-off, as a sysadmin > I'd be invited in and the developers and I would hash out > what versions of each language and library involved the > project would use To wit: the new ideologies in infrastructure management are actually designed to solve the underlying problem that necessitated that kind of working setup. Why should the version of a lib in one part of the software someh…

Counter-wit: Why can't people get their software to work with the existing libs? Hint: It's very rarely that the existing libs disqualify.
Post reply on HN