Earlier quoted context omitted.
I guess I come from a different background, work for different companies and have dealt with different problems. I am aware there is plenty of room for the traditional sysadmin in enterprise still but I am not quite sure how long those days will be around. Google and facebook where first to discover the "Site Reliability Engineer" but they were far from the last. SRE is becoming the new sysadmin and you can bet your…
I was doing "SRE" level system engineering at Amazon back in 2001 long before facebook ever existed. Also when I grew up as a junior system engineer in the mid-90s I looked up to "system administrators" like Wietse Venema who coded in C and wrote tcp_wrappers and postfix, and Larry Wall who wrote his own programming language. The System Administration / Software Developer is not a new invention of Facebook/Google. Wh…
Ask HN: How do I learn how to become a good sysadmin?
71–80 of 124 posts
Re: Ask HN: How do I learn how to become a good sysadmin?
#72Earlier quoted context omitted.
I'm a sysadmin for 16 years, I have worked in big and small companies and I don't agree with a great part of this post. I'm not a developer, I have never needed to disassemble or fix programs built by others, never needed to understand kernel subsystems or anything else kernel related (except may be how to replace a broken driver/module). I know nothing of C, I know just the basics of system calls and I've never hear…
Absolutely agree. Sysadmins' interface with code is basically to be a compliance-checker. Software, be it from in-house developers or a third party solution need to meet that compliance bar, or it doesn't go to production. If your in-house developers are unable to meet those compliance requirements, that's a them issue and not a you issue. It's good for devs to know some ops, and ops to know some dev from a familiari…
The companies that do this will have a velocity that outstrips the companies that do what you describe and will naturally have an advantage in the marketplace.
There will still be room for security compliance and someone in the company to be responsible for that, but they'll be writing tests that check software for compliance automatically, they won't be some drone in an office looking at a form and stamping their approval on a change request.
Your industry is changing. You may retire before it really hits you in the ass and kicks you out, but your advice for people getting into the industry is terrible.
Re: Ask HN: How do I learn how to become a good sysadmin?
#731) It's great that you ask and seek to learn to do it right; that's a first step many don't get past.
2) There are big differences between hacking on something at home and professional system administration. Most important is cost: Your time is expensive but system downtime can be incredibly costly. [1] You need to anticipate and prevent the problem in the first place (you are the expert!), have resources prepared in case of failure (including expert knowledge of the system), and resolve it quickly. Also, you are paid as an expert to get results that boost the bottom line. You can't spend a day fiddling around with something and gratifying your curiosity.
3) There is a very wide range of knowledge and skill among sysadmins. You don't need a license to do it; anyone can print "System Administrator" on their business card. There are many, many ignorant hacks; lots of decent ones who don't think beyond what they are told; and few true experts. Who you surround yourself with will determine, to a great extent, where you fall in that range. You probably will adopt their standards and you will learn their way of doing things; you can spend your time learning either the knowledge and techniques of the hacks or those of the experts.
4) Invest the time and effort to learn core technologies [2] exhaustively and to learn best practices. Never shy away from difficult technical material; push yourself to find the best sources and develop skill to understand complex material. Most of what's on the Internet is bullsh-t from and for amatuer hackers, good enough for your home server. You can spend your career without understanding much of what you are doing -- there's enough work out there for the hacks too. Find the very best sources and people (see #3), and take the time to learn from them. Learn something once and it pays off forever.
5) Learn to solve problems. Worry less about learning techniques (e.g., arcane details of command line switches); anyone can look those up. The real challenge is staring at a screen, seeing something that you have no idea about (and which is not in books or on the Internet), and finding a way to solve it (with all the time and other pressures mentioned above). To do that, you need a model in your mind, a deep understanding, of the technologies and systems involved (see #4).
6) Put yourself in situations where you will be in command of the situation and with time in reserve for making major enhancements, learning more, and for unexpected crises; not where you are struggling to keep up with a flood problems, frantically treading water (or slowly drowning). Also choose situations where you are prepared to handle the worst-case scenarios: When the sh-t hits the fan -- when all your plans and normal operations go to hell -- people will look to you to save them. Be their hero. (See #5.)
It can be a very intellectually stimulating and gratifying job. In every field, good people and true experts are hard to find. Make yourself into one and you will be in good shape.
---------
[1] Consider the cost of 1,000 people not working (avg hourly rate * 1,000 * hours), hours of orders missed, facilities shut down because your system is the bottleneck, deadlines missed, angry customers, embarassment to the business and its executives, etc.
[2] What the core technologies are for you will vary. You can't know everything. Beware of investing time learning about technologies that change quickly. TCP/IP probably will stick around for awhile; the app-of-the-week, maybe not.
Re: Ask HN: How do I learn how to become a good sysadmin?
#74Earlier quoted context omitted.
Would this list of books be a good starting point? http://everythingsysadmin.com/books.html
That is insufficent, try going through TCP/IP Illustrated Vol 1 at least and really learning TCP/IP and use tcpdump to rip apart packets. You should learn in detail the tcp state diagram and how to read netstat output. Know what the TCP 3-way handshake is and PTMU discovery and 4-way connection teardown and be able to apply that knowledge to debugging. Learn how to program in C at least well enough to write simple cl…
Re: Ask HN: How do I learn how to become a good sysadmin?
#75Sorry to disagree with a lot of posts here. >At the most basic level, when firing up a new server I follow the how to harden your server guides and install fail2ban, disable root login, enable ssh only login etc. This is simply wrong. There are two things wrong with this approach: 1. If I understood correctly, you are essentially following random guides on the internet about setting up your security rig. Not too far…
Following random guides on the internet doesn't necesarily have to be harmful if you don't simply copy-paste, but rather make an effort to understand what is the advice being given, why is it being given, and form an opinion about it. For instance, assuming you start from the very beginning, if a guide suggests disabling root login and you do your own research to understand what root accounts are, what can they do, a…
The best argument I could find for "disable root login" was "the attacker has to guess the username, too", which doesn't align with Kerckhoffs's principle and isn't the way security should be done, imo.
Also, fail2ban is a protection against bruteforce. If bruteforce is an issue for you, you're doing something wrong.
Please correct me if you know more.
Re: Ask HN: How do I learn how to become a good sysadmin?
#76Sorry to disagree with a lot of posts here. >At the most basic level, when firing up a new server I follow the how to harden your server guides and install fail2ban, disable root login, enable ssh only login etc. This is simply wrong. There are two things wrong with this approach: 1. If I understood correctly, you are essentially following random guides on the internet about setting up your security rig. Not too far…
You are right with the following blog posts blindly is a bad idea. And I feel guilty that I didn't invest time to learn why I commented those lines in IP conf (it's bad enough that I don't remember which files I changed). So I've definitely got a place to start working on already. The mentorship part is a lot more difficult. Good sysadmins are really difficult to find in Sri Lanka. I've worked with a few companies so…
Congratulations, it looks like you are one of the top sysadmins at your country.
Maybe you should try to look in other places, or maybe you really should really congratulate yourself, with no sarcasm at all, and start selling yourself like an expert.
Re: Ask HN: How do I learn how to become a good sysadmin?
#77 Usenix, https://www.usenix.org/
LOPSA, https://www.lopsa.org/
Then read the journals, even the articles that don't interest you now, and follow the email lists.Re: Ask HN: How do I learn how to become a good sysadmin?
#78Re: Ask HN: How do I learn how to become a good sysadmin?
#79Re: Ask HN: How do I learn how to become a good sysadmin?
#80Sysadmin is (unfortunately) a role that is on the decline, so from a pure employability perspective I'd suggest you focus more on the dev side. As for the skills, I'd suggest running a Linux distro as your personal, everyday machine, not just a server you log into on AWS or DO every so often to configure (which, also -- don't do that. You don't want snowflakes in your environment). It'll force you to learn a lot abou…
Why would you say that the need for Sysadmins is on the decline? If anything, I should think it would be greater than ever. I'm very curious here.