Live data from Hacker News

Ask HN: Do other “DevOps engineers” feel little skill/career progression?

news.ycombinator.com

1–5 of 5 posts

Ask HN: Do other “DevOps engineers” feel little skill/career progression?

#1
I've been working as a "DevOps engineer" for about 5 years now. While that title is often ambiguous, for me, it has mostly involved infrastructure work, mostly in AWS, automating deployments, managing CICD infrastructure, on-call rotations, etc.

I came to the realization recently that I haven't really felt like I'm working towards mastery of any skills and it's making me depressed. I know how to use a lot of tools, I've gained quite a bit of familiarity with systems end-to-end, but I generally feel like I learn just as much as I need to help solve problem at hand and then switch to something else. I feel like I'm always settled into this role of implementing minor improvements, creating proof-of-concepts, and helping people with infrastructure issues (aka being distracted).

I rarely get the feeling that I'm in any sort of good work "flow", or that I'm doing anything really useful, rather that I'm just cleaning up and making incremental dents in backlogs of tech debt.

I'm not sure if I'm not being assertive enough with my needs and ideas, or I find roles that are destined to continue this trend, or if I need to get out of "DevOps" engineering all together.

Does anyone else feel stagnated like this in their careers?

Re: Ask HN: Do other “DevOps engineers” feel little skill/career progression?

#2
Oh hey, it's something actually relevant to my interests. Seasoned DevOps Profeshnal hat goes on...nnnnow.

> I came to the realization recently that I haven't really felt like I'm working towards mastery of any skills and it's making me depressed.

"DevOps engineers" get the short end of the stick on the wide-vs-deep thing because "DevOps", at present, is constantly changing. And by "changing," I mostly mean "thrashing," though we have found a decent local maximum with some of the container stuff that's out there that I think will last, even on the leading edge, for at least a few years.

Which is to say, this is endemic. Best way out, IMO, is to have other professional outlets. I write a lot of not-DevOps code, both at my current gig and on the side, because DevOps requires the Dev side as well as the Ops side to be worth a fart.

> and helping people with infrastructure issues (aka being distracted). ... I rarely get the feeling that I'm in any sort of good work "flow",

Interruptive time/distractions are a universal problem. They're never going to go away. What you can and should do (and this requires a manager who Gets It, so, y'know, YMMV) is set up office hours for those interruptions. "I am available between X and Y on days A, B, and C to help with your DevOps problems." Obviously this doesn't apply to stuff like production outages, but it does apply to stuff like "how should we best accomplish this task?" or "I need this thing run against the dev environment."

Bonus points, for the last one above, if you can use those requests to build data to present a business case upward. If you get a lot of requests to do one or two related things, you can use that data to push your boss to find time for you automate it (or to find developer help for it, whatever).

> rather that I'm just cleaning up and making incremental dents in backlogs of tech debt.

There are two kinds of DevOps engineering, in my experience: the first one and the second one. No, that's not right--janitorial and architectural. You (and I'm cheating a bit, 'cause I know who is really beneath that tremendous hat) are stuck with a lot of the janitorial stuff because, I think, you tend to come into established shops that have a Way Of Doing Things and, incidentally, don't offer a lot of ways to move upward. The architectural roles at companies like that tend to already be filled (and DevOps, while not hierarchical, does seem to tend to concentrate a lot of the architectural decision-making to people like...well...me).

Smaller companies might benefit you, in that you get to take a big swing or two at the full stack of problems.

> I'm not sure if I'm not being assertive enough with my needs and ideas

You're not, but most people aren't. Big personalities (hi) tend to fill the space. You don't have to be a jerk, but you do have to kinda push to get what you want to be yours.

Oh, and get out of owning CI/CD. Make developers own that stuff. You'll be happier. They won't be, until they actually learn what they're doing, but, y'know. It's good for them. ;)

Re: Ask HN: Do other “DevOps engineers” feel little skill/career progression?

#3
I have a few scattered thoughts about this, and I'm not sure I have a single thesis in this post, but I'll type up some of my experiences and hopefully it'll be helpful in some way. I have also been doing DevOps for 10-100 person engineering orgs for around 5 years, and I have also felt stagnant, interrupt-driven, and generally frustrated by perfectly capable engineers waiting on me to make them a database while I'm busy trying to keep their applications executing on computers.

I'll apologize in advance for talking about myself a lot in this post, I'm not sure exactly where you're at career-wise, and I don't like to be prescriptive about generalities, but hopefully something about my own experience can be helpful to you or someone else.

Thought 1: Norms become exponentially harder to change the longer they exist.

I did a poor job of setting the right norms when I started my previous job, and it resulted in a completely unsustainable environment where engineering was 100% focused on development goals, and I was responsible for keeping the product afloat. I consider this my mistake, because I was the one explicitly hired to handle that problem, and the way I handled it was by creating a single point of failure - me.

I don't consider that job a complete failure, I delivered on the things I was hired to build. But by the time I realized my mistake, it was already The Way Things Are Done. I didn't have the skills to change it, or even the words to explain it yet.

These days I generally try not to ops anything I didn't dev, and I expect the same from my colleagues from day 1. When someone asks me for a database, I say "Sure, let me come sit with you and we can walk through creating one." Which leads me to thought 2:

Thought 2: It's harder to teach someone to fish than it is to give them a fish.

But the corollary about eating for a lifetime is still true. It's usually faster for me to spin up a new database, but if I do it, I'm robbing the team of a chance to learn how their new database works. I am neglecting my responsibilities as a senior engineer when I don't take the time to teach.

I've been at my current employer for about a year and a half, and it's paid off - engineers routinely write infrastructure code, build their own containers and CD pipelines, and deploy new services to production. They use some of my tools to do it, and I make sure to take the time to help them be successful with them, but I make sure that they're the ones pushing the buttons. The payoff is in quotes like this, which one of my colleagues shared in our engineering channel today (paraphrased):

> This is long overdue but I wanted to give a shout out to @edwards for the ops tooling!!! I was able to deploy a new API (staging AND production) in minutes which, if I did manually, would have taken days. Thanks for iterating and giving us the tools to build things that Just Work. I have easily spent days on this in the past, especially things like connecting load balancers to services. And, with Cloudformation, we know we can easily recreate environments in case something goes wrong. Amazing stuff!!

Thought 3: Soft skills are harder than hard skills

No part of a computer is magic, and at some point you just sorta figure out how they work, and the mystery and awe of it fades out completely, and the job you're left with is giving rote instructions to an army of perfectly obedient, _ruthlessly_ pedantic children. They understand nothing and they don't sleep.

The difference between a junior engineer and a mid-level engineer is the breadth and/or depth of their technical knowledge, but the difference between mid-level and senior is how well they communicate that technical knowledge, and how they collaborate with others. This doesn't necessarily mean management, but it does mean a big adjustment in the coding/collaboration ratio. (These days I probably delete as much code as I write...)

One of the biggest changes I've made over the past year or two is in how I view my own responsibilities. I am not responsible for production, that isn't sustainable, and it doesn't scale. I am responsible for making sure _the entire engineering team_ is empowered to handle production. I made that decision when I started my current job and it's paid off tremendously.

Re: Ask HN: Do other “DevOps engineers” feel little skill/career progression?

#4

I have a few scattered thoughts about this, and I'm not sure I have a single thesis in this post, but I'll type up some of my experiences and hopefully it'll be helpful in some way. I have also been doing DevOps for 10-100 person engineering orgs for around 5 years, and I have also felt stagnant, interrupt-driven, and generally frustrated by perfectly capable engineers waiting on me to make them a database while I'm…

A lot of what you wrote resonates with me, particularly the comments about soft vs hard skills. I think I need to seek more opportunities in my organization to teach and collaborate, because those interactions generally make me feel much better and more valued than whatever I produce code-wise. Perhaps those interactions are what can cascade down towards promoting self-sufficiency on the dev teams and good norms (your other points).

I appreciate your thoughtful response, thank you for taking the time to write it up!

Re: Ask HN: Do other “DevOps engineers” feel little skill/career progression?

#5
post #2

Oh hey, it's something actually relevant to my interests. Seasoned DevOps Profeshnal hat goes on...nnnnow. > I came to the realization recently that I haven't really felt like I'm working towards mastery of any skills and it's making me depressed. "DevOps engineers" get the short end of the stick on the wide-vs-deep thing because "DevOps", at present, is constantly changing. And by "changing," I mostly mean "thrashin…

Thank you for the response Ed, it's crazy that you stumbled upon this post without any outside influence whatsoever ;) I'm always eager to hear your perspective on things.