Live data from Hacker News

Software Developers Should Have Sysadmin Experience

blog.professorbeekums.com

81–90 of 424 posts

Re: Software Developers Should Have Sysadmin Experience

#81
post #76
post #25

Earlier quoted context omitted.

I think about frontend designers (User experience people). They should know a little bit about TCP, SSL, HTTP, networking, monitoring, statistics, and everything related to frontend markup bloat, bandwidth, compression, etc. I've work with many of them, very good about Adobe products and Apple last thing, but clueless about, why their final code is of poor quality. Fortunately, many of them did get their Ego down, wh…

Agree with most of the list but why is TCP important for frontend developers? I'm struggling to find the arc for that one that isn't abstracted away much like CPU pipelines are for back end devs.

I prefer to work with a frontend engineer that knows a little bit about packet loss, routing, load balancing, white/black listing, retransmission, packet fragmentation, latency, mobile connections, socket limits, common error messages on network diagnostics, etc... that with an engineer that just knows "the page is slow".

Well, indeed, the higher level protocols, like HTTP, DNS, etc, are more useful on a daily basis.

Re: Software Developers Should Have Sysadmin Experience

#82
post #76
post #25

Earlier quoted context omitted.

I think about frontend designers (User experience people). They should know a little bit about TCP, SSL, HTTP, networking, monitoring, statistics, and everything related to frontend markup bloat, bandwidth, compression, etc. I've work with many of them, very good about Adobe products and Apple last thing, but clueless about, why their final code is of poor quality. Fortunately, many of them did get their Ego down, wh…

Agree with most of the list but why is TCP important for frontend developers? I'm struggling to find the arc for that one that isn't abstracted away much like CPU pipelines are for back end devs.

A front end developer should understand that a TCP request (i.e what an HTTP request is) consists of a handshake which means to receive a file from a server, it takes roughly 5/6 calls to and from the server. So if your UI consists of 1000 100b files, that's 6000 calls. So you would probably be better concatenating them to a single 10Kb file.

But then you need to understand the differences between HTTP 1.1 and 2.0 and how it handles multiple requests from the same page. 1.1, you would probably be better off with a single concatenated file, 2.0, perhaps several. And then what about compression? Any good UI developer should be considering compression of that file, cache control, etc.

I see too many front end developers shy away from this stuff and create bloated monstrosities. They feel it's someone else's job... but if not the UI developer, then who??

Re: Software Developers Should Have Sysadmin Experience

#83
post #65
post #50

This might be controversial, but I don't think you get to be a half decent developer without being a reasonable sysadmin. Maybe my experience is unusual, but I've never worked anywhere that the sysadmins knew more than the developers about how best to run their code in production. And when things go wrong with it how best to find the cause of the issue. And I've never thrown code over a wall without having tested it…

>>> The worst sysadmins get in the way of developers. Ones that scale down your CI server to the cheapest, throttled, one the hosting company has, leaving $800/day contract developers waiting for builds that run in 20 seconds on their laptops take nearly an hour. How likely is that the sysadmins were told to 'just make it run cheaper, I don't care' by someone higher in the foodchain?

How likely is that the sysadmins were told to 'just make it run cheaper, I don't care' by someone higher in the foodchain?

If you're blindly following "orders" to reduce costs and doing things that push up costs elsewhere then you're not doing a good job. A good sysadmin (or the sysadmin's boss) should be able to pull up some numbers and say "Build tasks are being queued for an hour before they run. What impact is that having?", and call a wider meeting that brings together the higher-up-the-foodchain manager, the development team, and anyone else who might be affected. Ideally it'd be the higher up manager who calls that meeting of course, but they may not understand the technical issues.

Re: Software Developers Should Have Sysadmin Experience

#84
post #53
post #39

Earlier quoted context omitted.

I've worked with all kinds - from windows devs who can't figure out how to install visual studio - to people who understand Windows, Linux, and macOS - as well as basic system administration for each platform. The people who are most successful at rapidly developing good high quality software are more in the later group. Would you trust a RF engineer who couldn't troubleshoot his own radio designs? why would you trou…

In fairness to the folk who can't install Visual Studio - it's a genuine pain in the butt. Last time I tried, half of the download links were absolutely non-functional. Their documentation didn't help much, either, since they pointed to the non-functioning links. I got a lot of shit for that one, but felt completely vindicated when it happened to someone else a year later.

> In fairness to the folk who can't install Visual Studio - it's a genuine pain in the butt.

It takes a while but I've never found it to be a pain in the butt and I've used it, off and on, since version 6 in the late 90s.

If the install errors out it gives you an error message, you google for it, figure out what the problem is (most common messages are easy to find solutions for), fix the problem, reinstall, done.

Re: Software Developers Should Have Sysadmin Experience

#85
post #19

Earlier quoted context omitted.

The difference with this is that server-side software has direct operational consequences. A server-side developer who never deals with ops is like a chef who never tastes the food. It's in theory possible to get right, but in practice the results tend to be poor.

That's true for backend, but frontend developers don't necessarily need a lot of ops knowledge if the team has a good separation of concerns. If frontend devs have to worry much about sys admin issues, I'd say that likely points to a flaw in the way things are being done. Of course, the more someone knows, the better. Knowledge and experience in any area of technology can improve understanding of all the others. But…

> [...] frontend developers don't necessarily need a lot of ops knowledge if the team has a good separation of concerns.

If there is a sysadmin in your development team and this sysadmin has something to say about system's architecture and development workflow, then OK, you don't need any of your programmers to be a decent sysadmin. Otherwise, you need to get the sysadmin.

> But nobody has time to learn everything, and there's way too much to learn. Time given to learning more about sys admin issues is time lost to other potentially valuable knowledge.

And then we have Homebrew, which I constantly hear is atrocious and breaks software on updates (incorrectly managed dependencies), and its developers don't understand the necessity of using cryptography for transporting packages. We also have this whole idiocy of deploying software in production by recompiling it with `pip install', `rbenv install', and `npm install'. And then we also have incorrectly built RPM and DEB packages published by software developers (InfluxDB and ElasticSearch are common offenders, Riemann being another one when I checked long time ago).

And mind you, I don't use MacOS, I don't develop web applications, and I package and host additional software myself, so how big are these issues when they got my attention? What was it again with your valuable developer's learning time?

Re: Software Developers Should Have Sysadmin Experience

#86
post #74
post #50

This might be controversial, but I don't think you get to be a half decent developer without being a reasonable sysadmin. Maybe my experience is unusual, but I've never worked anywhere that the sysadmins knew more than the developers about how best to run their code in production. And when things go wrong with it how best to find the cause of the issue. And I've never thrown code over a wall without having tested it…

I've done the dual sysadmin/developer thing for a small company, and the problem I experienced there was completely incompatible working modes. Sysadmins must deal with interrupts (requests, crises, things driven by external schedules etc) and then in the rest of their time build systems to manage or reduce the interrupts. Developers are expected to produce work on a predictable schedule. This is disrupted by interru…

I had exactly the same experience at a previous employer, almost word-for-word.

Having said that, the central assertion is still correct: the absolute best developers I've ever worked with were also top-tier sysadmins (or linux experts, depending on what you want to call it).

Re: Software Developers Should Have Sysadmin Experience

#87

Earlier quoted context omitted.

That question doesn't make any sense at all. That's exactly the same as saying: with the rise of do it yourself home repair, how often are plumbers different from carpenters?

Not the same at all. Let me rephrase the question. Reading the article, the sysadmin is described as the person who constructs and maintains the environment the code runs in, including the physical machines, but also things like dependency management (I infer). Given that just about everything except the physical machine can be handled through devops automation (from experience), and those things don't require "speci…

I started as a dev many years ago, then moved into sysadmin / devops stuff for over a decade, and then moved back to more hands on dev stuff again a few years ago. I think your statement that "those things don't require "special" skills other than docker/ansible knowledge" is over-simplifying things a bit.

Writing a production quality Dockerfile or Ansible Playbook requires more than just a bunch of "apt-get install" statements. You need a bunch of knowledge about how each bit of software should be configured and tuned for each individual use-case.

Once it's been written, then the devs can happily automate all the things as much as they like. Writing it in the first place is where a lot of what was traditionally sysadmin knowledge comes into play.

Anyone with a bit of patience and practice can follow a recipe in a cook-book. Not everyone can write one.

Re: Software Developers Should Have Sysadmin Experience

#88

Should sysadmins have software development experience (e.g. DevOps)? For what values should 'X have Y experience?' Should we go as far as... "A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new proble…

I strenuously disagree with some of these.

> Plan an invasion

This is actually a massive undertaking. An undergraduate at MIT taking a semester-long course on this will barely scratch the surface of it. Furthermore, you're never going to suddenly and unexpectedly need to know this. Any situation where you plan an invasion is going to be preceded by spending a long time getting into the position where people trust you with their lives and the fates of their nation.

> die gallantly

You're only ever going to be in this situation once, and probably not even that. Why does it matter how gallant your heart attack is?

These skills make a bit more sense in a world where most of us need to march off to war. Happily, we don't live in that world.

You should prepare for the situations you are only mildly unlikely to be in and where your skill matters.

Re: Software Developers Should Have Sysadmin Experience

#89
post #20

We took it one step further on my team. The best way to ensure that the pager(duty) doesn't go off at 2am in the morning is to put developers in the first on-call group. Not only did it work, but the developers got a much more nuanced view of how systems operate. When we first started, I'd see developers using ping to determine whether a nameservice entry was correct. After a few months of handling almost all of our…

I hope you increased their salaries together with the new 2am pager duty.

Re: Software Developers Should Have Sysadmin Experience

#90
post #2

Somewhat related, I feel that mechanical engineers who design cars should have some experience servicing vehicles. Sometimes engineers will not leave enough space, use weird fasteners, etc. that make a simple job much more complex.

It could be a deliberate form of obfuscation/discouraging of repair, or just the common trend of making things more complex than they really need to be.

The BMW i8 is a good example https://www.youtube.com/watch?v=fxe_b2GRwok
Post reply on HN