Software Developers Should Have Sysadmin Experience • Hacker News Board

Re: Software Developers Should Have Sysadmin Experience

#361

Post by AnonymousPlanet » Mon, Jan 09, 2017, 8:59 PM UTC

just2n wrote: post #354

Anecdotal as it is, I started my career working as a sysadmin while studying CS. When I was younger I was quite interested in security (e.g. I followed defcon and CVE lists and read a lot of manuals and source code). I built server software, broke applications, and did a lot of reverse engineering. In that time I was forced effectively to learn about how the OS worked and how to utilize it, both Linux and Windows, fr…

I have a very similar story as a background. To add to what you said: while maintaining and debugging software installs, I learned a lot about how and when things break. Especially, how important it is to keep things simple. This turned out to be invaluable when I began working as a software developer after graduating.

Re: Software Developers Should Have Sysadmin Experience

#362

Post by nasalgoat » Mon, Jan 09, 2017, 9:45 PM UTC

brianwawok wrote: post #240

jstimpfle wrote: post #211

Earlier quoted context omitted.

Another perspective: Why don't you want your software to be compatible with the system that your sysadmin provides? (Assuming that system is not completely obsolete). Minimize your dependencies. It's incidentally also what leads to clean code bases.

> Minimize your dependencies. It's incidentally also what leads to clean code bases. You also crush velocity. Smart use of libraries lets you ship code 10x faster. Two identical businesses.. one writes all their own code, one is smart about using libraries. Which one makes it to IPO first, and which goes bankrupt?

In my experience, the vast number of problems we've had with our mobile apps, especially on Android, has been a developer deciding to use some random SDK to solve a simple problem because he didn't want to take the time to write it himself.

Then the app is broken or has memory crashes, or the final binary is 10x the size it needs to be. 9 out of 10 times it's a third party library. This is why I ban the use of them unless absolutely needed.

Re: Software Developers Should Have Sysadmin Experience

#363

Post by vickychijwani » Mon, Jan 09, 2017, 9:51 PM UTC

I'm moving from dev to devops for a while, because of similar reasons. Excited to learn all these new things and peer beneath all the tools and abstractions I rely on! I have no doubt in my mind that it will make me a better developer - most abstractions have a tendency to leak, eventually.

Re: Software Developers Should Have Sysadmin Experience

#364

Post by rcymerys » Mon, Jan 09, 2017, 10:07 PM UTC

eviln1 wrote: post #133

Hi, I'm a Sysadmin, and I've been a grumpy one through a larger part of my 15 years experience. My main issue was that Developers were acting like Users: they don't care about what you have to deal with, they want things to 'just work'. In return, I've treated them like children, in some instances yelled at them when they did dumb stuff. I've tried to educate them when possible, and was angry when the education didn'…

I couldn't agree more.

I didn't realize the importance of all the "admin stuff", before the our newly hired sysadmin came to me and asked if I could help him figure out how to deploy the project I was working on. This ended up being a looong chat about monitoring, redundancy, architecture, security... you name it. What I've always thought of as installing and configuring software turned out to also touch designing the software so that it works reliably and is easy to maintain.

I don't think I'll ever have plenty of sysadmin skills, but knowing even the general idea of what's important to sysadmins helps a lot. Also, being able to become another interruption in their day and consult ideas is priceless. :)

Re: Software Developers Should Have Sysadmin Experience

#365

Post by CaptSpify » Mon, Jan 09, 2017, 10:13 PM UTC

jasonlotito wrote: post #331

mancerayder wrote: post #323

Earlier quoted context omitted.

Those technologies don't exist so Developers can get around Sys Admins and ignore your helpful advice. They exist to solve that problem that makes the Sys Admins role there necessary. It removes the underlying need for a Sys Admin to worry about the versions. Admins should see this as a good thing, but in my experience many dislike it because it takes them out of their Gatekeeper role. We shouldn't WANT to stop the D…

> However, we "Gatekeepers" are the ones that get paged and / or yelled at by a CTO when an application keels over. Not the developers. And that's why people are moving away from that model. It's part of the reason DevOps is being embraced as a model. Developers should be on call to support the applications they build. You get benefits all around.

should is the key-word here. As a sysadmin, I'd love to work closer with devs, especially during outages. Unfortunately every time we bring up on-call, the room goes silent. This is very anecdotal, but IME, devops has just become a way for devs to bypass sysadmins.

I wonder how many companies are doing it right vs doing it wrong? Any anecdotes from a proper devops group?

Re: Software Developers Should Have Sysadmin Experience

#366

Post by CaptSpify » Mon, Jan 09, 2017, 10:16 PM UTC

serge2k wrote: post #338

mancerayder wrote: post #323

Earlier quoted context omitted.

Those technologies don't exist so Developers can get around Sys Admins and ignore your helpful advice. They exist to solve that problem that makes the Sys Admins role there necessary. It removes the underlying need for a Sys Admin to worry about the versions. Admins should see this as a good thing, but in my experience many dislike it because it takes them out of their Gatekeeper role. We shouldn't WANT to stop the D…

and we get yelled at when we can't deliver a feature because some gatekeeper is sitting atop his little throne in the kingdom of servers saying no. ;) This is institutional failure.

Then place the blame on the gatekeeper. As a sysadmin, I'd be more than happy with you pointing the finger at me as the reason why you can't deliver a feature. Assuming, of course, that you've run the proper tests and gotten QA's approval.

Re: Software Developers Should Have Sysadmin Experience

#367

Post by user5994461 » Mon, Jan 09, 2017, 10:23 PM UTC

mattmanser wrote: post #344

BinaryIdiot wrote: post #343

Earlier quoted context omitted.

> Then you were lucky. I worked in 3 Microsoft dev shops and 1 that had a mixture. As far as I know I hadn't heard of anyone having issues getting it installed except for the rare, occasional error that could be Googled and fixed. I'm not sure I'd call that luck, sounds like you just had a bad experience. But yeah back in the day it was a stack of discs (I think the last disc version I used had 2 discs for visual stu…

There were 40 DVDs in a MSDN subscription in 2004, so not sure what you're talking about, you might have had the cheapo one?

I've worked on VS since VS 2003.

The first versions (200X) had some challenge at times. Starting from locating the installer for the right edition and the license. Then minimal setup to have a working environment was split in 5 different installer/projects to be executed in orders (one VS pack per language + the Windows SDK + the debugger kit + the ATL/MFC package + the driver kit [if you dev drivers] + the DirectX SDK [if you need it]). Then configure some PATH and libraries to link together all of that.

Last I checked, in 201X editions. A lot have been regrouped in a single setup. That's enough for most developments. And the optional packages have auto detection (and it ain't fucked it you run it twice).

Re: Software Developers Should Have Sysadmin Experience

#368

Post by eropple » Mon, Jan 09, 2017, 10:29 PM UTC

peterwwillis wrote: post #318

eropple wrote: post #67

Earlier quoted context omitted.

I've read your post two or three times now, and I straight-up don't get what you're driving at. Yes, there are levels of subject matter expertise that you need to know--or at least be conversant enough with to deep-dive when necessary--for system administration. That's also the case for database programming or bioinformatics or graphics programming or whatever. But it is still expressed through the writing of code ,…

I would be horrified if you had to be a software developer to be a sysadmin. It would be like requiring an auto mechanic to be an engineer, or a cook to be a chemist or something. In actual fact, the complexity of software design is antithetical to the job of system administration. If at all possible you should use the least code possible to do a given job, and rely on the reuse of tools to serve various functions. T…

> If at all possible you should use the least code possible to do a given job, and rely on the reuse of tools to serve various functions.

I agree, on both counts--and you should write the least code possible and reuse the most tools possible when you're building a web app, too! And when I am managing systems and infrastructure, I am reusing tools that are configured and operated through the expression of domain-specific code in Ruby. When I'm solving a problem, I'm writing code in Chef or in Cfer/CloudFormation (if I'm touching a machine by hand that's outside of my burn-it-down test environment, something's going really, really wrong). I'm consciously writing that code in such a way that I can put it into an automated test environment based on preconditions and postconditions that map to the business logic of the system to ensure correctness when that code is then changed. If I need to solve a related problem later, I'll generalize the code for least-effort reuse at that point.

This is literally software development, to me. That the output involves a configured system instead of a web app or whatever is orthogonal to being software development. It still feels like you are drawing a distinction without a difference.

Re: Software Developers Should Have Sysadmin Experience

#369

Post by eropple » Mon, Jan 09, 2017, 10:37 PM UTC

schnable wrote: post #167

swozey wrote: post #138

Earlier quoted context omitted.

When I was getting into ops 10 years ago the book "The Practice of System and Network Administration" by Thomas Limoncelli was a great overview (very high level and not specific) that we all read. The Phoenix Project by Gene Kim is kind of a more modern uptake on it, but both can be read easily by people outside of the field and I doubt the first one will ever not be relevant. Both are more about establishing proper…

Limoncelli's newer book _The Practice of Cloud System Administration_ is awesome. It's really more about how to build distributed systems to performant and operational.

I started writing a book about cloud system administration before I read this one, FWIW. It's worth your time.

Re: Software Developers Should Have Sysadmin Experience

#370

Post by tetha » Mon, Jan 09, 2017, 10:38 PM UTC

ryandrake wrote: post #347

cableshaft wrote: post #249

Earlier quoted context omitted.

> I half believe the reason developers are embracing cloud architecture so much is to remove so many sysadmins out of the equation. It was literally true at one of my previous jobs. We couldn't install anything on our own dev machines without approval from Net Ops, not even Notepad++ (I don't think I ever got that installed, never got approval). We once asked for a new server which mirrored the software of an existin…

I've seen this as well. "Timeline from internal IT for provisioning a box and deploying our app is 6 months, and subsequent changes go through a ticketing system with a 2 week average turnaround. Or, we can have it running ourselves on AWS in 30 minutes."

Shit. So, you're saying that my mess of a team is kinda awesome by implementing reliable production-ready deployments within 2-4 weeks, and implementing changes to environments within like 30 minutes ("set key foo to bar in configs please?") to a week ("we need persistence!").

I guess IT in this place really is getting up to speed.