Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software Developers Should Have Sysadmin Experience

blog.professorbeekums.com

Post reply on HN
Print view View original

221–230 of 424 posts

Re: Software Developers Should Have Sysadmin Experience

#221

Post by wpietri »

danenania wrote: post #36
wpietri wrote: post #19

Earlier quoted context omitted.

The difference with this is that server-side software has direct operational consequences. A server-side developer who never deals with ops is like a chef who never tastes the food. It's in theory possible to get right, but in practice the results tend to be poor.

That's true for backend, but frontend developers don't necessarily need a lot of ops knowledge if the team has a good separation of concerns. If frontend devs have to worry much about sys admin issues, I'd say that likely points to a flaw in the way things are being done. Of course, the more someone knows, the better. Knowledge and experience in any area of technology can improve understanding of all the others. But…

> but through good architecture and management practices, there are still plenty of ways to make very high value contributions for a developer who hasn't spent much time focusing on sys admin.

Huh. Hands up everybody who is working at a place with good architecture and management practices?

Knowledge of and experience with the direct downstream and upstream of one's work is different than general knowledge. It can in theory be made up for by high-ceremony processes and strongly controlled interactions. But in practice that mostly doesn't work in software development.

Front-end developers don't need much server ops knowledge because that's not generally the direct upstream or downstream for them. But the principle still applies; they will be better if they have experience with their upstream and downstream.

Re: Software Developers Should Have Sysadmin Experience

#222

Post by web007 »

briffle wrote: post #181
Clubber wrote: post #154

Earlier quoted context omitted.

I agree. I also think it should work both ways. The worst jobs I've ever had is when the sysadmins have the mindset where they own the servers and are unwilling to deviate from what they've read at the behest of the developers. "I will force AV on reads on the developer boxes." "I will install AV on the production DB servers without telling anyone in the development group, then make the developers prove AV was the ca…

what your probably not seeing, is the CIO/CSO screaming at the SA to get AV deployed on every machine in the company, to meet some audit requirement checkbox, or PCI compliance, by the end of the month.

Exactly this. Audits don't care if there isn't any practical malware or if nobody can access the system outside 3306 and 22. Audits say "all production systems implement antivirus software" as a binary checkbox.

Re: Software Developers Should Have Sysadmin Experience

#223

Post by simplehuman »

jstimpfle wrote: post #220
awinder wrote: post #216

Earlier quoted context omitted.

> 15 years ago, when a project would kick-off, as a sysadmin > I'd be invited in and the developers and I would hash out > what versions of each language and library involved the > project would use To wit: the new ideologies in infrastructure management are actually designed to solve the underlying problem that necessitated that kind of working setup. Why should the version of a lib in one part of the software someh…

Counter-wit: Why can't people get their software to work with the existing libs? Hint: It's very rarely that the existing libs disqualify.

Because many of these old library versions go unmaintained.

Re: Software Developers Should Have Sysadmin Experience

#224

Post by ad-hominem »

fnj wrote: post #125
rtpg wrote: post #114

Earlier quoted context omitted.

I guess my point was more that even just if you understand HTTP then you'll get all those insights already. A bit pedantic I suppose. This stuff does get pretty complicated. Mainly because the tech is evolving fast (browsers do a lot of tricks now). Even in HTTP 1.1, there are times where multiple files might be more interesting due to how large files are handled

But can you really understand HTTP without understanding TCP?

Can you really understand js without understanding assembly?

Re: Software Developers Should Have Sysadmin Experience

#225

Post by monocasa »

jstimpfle wrote: post #214
cs02rm0 wrote: post #50

This might be controversial, but I don't think you get to be a half decent developer without being a reasonable sysadmin. Maybe my experience is unusual, but I've never worked anywhere that the sysadmins knew more than the developers about how best to run their code in production. And when things go wrong with it how best to find the cause of the issue. And I've never thrown code over a wall without having tested it…

> but I don't think you get to be a half decent developer without being a reasonable sysadmin I think you've just described me. And I don't see the argument against sysadmin. > leaving $800/day contract developers waiting for builds that run in 20 seconds on their laptops take nearly an hour. Sorry I just can't take you seriously.

Honestly, I've heard worse out of sysadmins in some places.

Re: Software Developers Should Have Sysadmin Experience

#226

Post by iask »

AlexB138 wrote: post #200
bandrami wrote: post #164

As a grumpy evil sysadmin, I think the good Professor misses where the real disconnect is, at least nowadays: stack management. Why do things like Docker exist? Because developers got tired of sysadmins saying "sorry, you can't upgrade Ruby in the middle of this project". Why does virtualenv exist? A similar reason. Containerized ecosystems (which is to say basically all of them now) are really a sign of those of us…

As a Sys Admin turned Automation/Tools Engineer, I think you're missing part of the point. You've got the beginning of it right in saying that Sys Admins used to be involved in pinning down versions, and even in why that was necessary, but I believe you're incorrect in saying that the containerization technologies are bad for removing that. Those technologies don't exist so Developers can get around Sys Admins and ig…

At what point do you determine who is responsible for securing the environment? The "Gatekeeper" mentality is stemming from this. There is no clear line in any organization and I see the blame-game all the time.

Re: Software Developers Should Have Sysadmin Experience

#227

Post by adrianN »

jstimpfle wrote: post #220
awinder wrote: post #216

Earlier quoted context omitted.

> 15 years ago, when a project would kick-off, as a sysadmin > I'd be invited in and the developers and I would hash out > what versions of each language and library involved the > project would use To wit: the new ideologies in infrastructure management are actually designed to solve the underlying problem that necessitated that kind of working setup. Why should the version of a lib in one part of the software someh…

Counter-wit: Why can't people get their software to work with the existing libs? Hint: It's very rarely that the existing libs disqualify.

People could do great things just with punch cards, yet somehow technology kept marching on.

If developers want to use newer stuff usually they have a good reason. The ability to hack around the deficiencies of old dependencies does not mean that one couldn't get a better, cheaper solution with newer technology.

Re: Software Developers Should Have Sysadmin Experience

#228

Post by bandrami »

btreecat wrote: post #215
bandrami wrote: post #188

Earlier quoted context omitted.

why should a sysadmin who doesn't even know Python tell me what version of a library I will use Because he maintains that installation and you don't? But, yeah, that's why virtualenv, Docker, etc. were invented, because devs kept getting sick of installations having consequences. What does a sysadmin have to approve or deny an upgrade? Check for conflicts of this version of this library with other software currently…

>Because he maintains that installation and you don't? So why is that an acceptable mentality for "in-house" developed software but if you buy something proprietary from a third party where you have zero say over what lib/langs are used, it's A-OK? >Check for conflicts of this version of this library with other software currently in use (by other developers maybe, or even by the same developer). That's not the case w…

I see your point, and like I said I agree this is why Docker was invented and it's the best-in-breed at what it does (namely, being a tourniquet for a self-inflicted wound). My biggest concern really isn't "my problem" since I'm in ops: it's the leftpad worry. I still have teams starting Dockerfiles with "FROM centos:latest" because that's just the mindset they have: "Latest will fix any problems" rather than "Latest will introduce new problems".

And, ultimately, Docker lets that not be my problem, because they have to deal with it when the next leftpad happens. So, yeah: they should have at it. I guess I still think there's something to be said for the cathedral pace, though.

Re: Software Developers Should Have Sysadmin Experience

#229

Post by hueving »

LoSboccacc wrote: post #119
cs02rm0 wrote: post #50

This might be controversial, but I don't think you get to be a half decent developer without being a reasonable sysadmin. Maybe my experience is unusual, but I've never worked anywhere that the sysadmins knew more than the developers about how best to run their code in production. And when things go wrong with it how best to find the cause of the issue. And I've never thrown code over a wall without having tested it…

> thrown code over a wall without having tested it this is the weirdest part of the whole devops mantra. like, I know how to evaluate the complexity and memory requirement of code way before I write it and I guess most compsci should be able to do the same. so either it's yet one more attempt in getting cheap labor into workable territory or plenty people where this myth originated are being cheated out of their mone…

>the complexity and memory requirement of code

Those are only tiny slices of real production bugs. No amount of complexity analysis of your code ahead of time is going to protect you from all of the issues that arise with integrating any large system dealing with lots of requests. You run into all kinds of things like query optimization, kernel TCP tuning, load balancer problems, cache thrashing, high latency clients, out of spec clients, power failures, etc.

If you think knowing the theoretical behavior of your program in an ideal environment is enough, you are exactly the type that throws code over a wall without having tested it.

Re: Software Developers Should Have Sysadmin Experience

#230

Post by wiredfool »

slavik81 wrote: post #68
userbinator wrote: post #9

Earlier quoted context omitted.

It could be a deliberate form of obfuscation/discouraging of repair, or just the common trend of making things more complex than they really need to be.

That's possible, but lack of insight into what's easy or hard in manufacturing or construction is a pretty common problem. I saw it a number of times when working for a few different manufacturing companies, though none of them built cars. A failure to understand construction concerns also played a role in the Hyatt Regency walkway collapse. The original design was poor, but redesign to address construction difficult…

There was another problem -- the beams were spec'd as box section, but what was on the shop drawings was 2 channel sections with the flanges welded together. (like this: []). The attachment point that supported twice the design load was also compromised by the weld and less competent section. Whoever was checking the shop drawings didn't pick up on the importance of the change.

IIRC, either change on it's own would have been marginally ok, the two together weren't. (by marginally, I mean probably wouldn't have killed people but wouldn't be to code)

When I was going through civil engineering, there was a big push to use a statistical basis for loads and resistances, rather than using a blanket factor of safety. Loads vary, strengths vary, potentially normally, probably not. But they're described by statistics at any rate. Blunders of this sort aren't, at least on a per project basis.

Post reply on HN