Live data from Hacker News

Why should I do production support?

devenbhooshan.wordpress.com

131–136 of 136 posts

Re: Why should I do production support?

#131

I like to write software but I don't want to be on-call if the software I wrote breaks at 3am in the morning. I do take my job with professionalism, I do write tests for most of it (not 100% coverage, but 100% coverage of the critical parts), I do monitoring (and answer and fix alerts if they happen during working hours) and I don't deploy on Fridays (and don't allow people to deploy on Fridays). My code will crash s…

This attitude almost always turns into the following: On-call: "Hey devs, I'm being woken up at 3AM because your app sucks. Please fix it." Devs: "Sure, no problem." 4 months go by On-call: "These alerts are still coming in at 3AM. Did you fix the issue?" Dev: "We have a lot of work, we can't dedicate all our time to some minor problems, we have a deadline." Next week, Devs are put on-call. The alerts are fixed in tw…

> This attitude almost always turns into the following: [...]

Well, that's another problem (the dev not being able to solve a bug that reappears at 3AM).

> The alerts are fixed in two weeks. Site reliability goes up. Apps suddenly become more resilient to failure.

I always wondered why DevOps has the "Dev" in its title. At least, in most of the companies I have worked on, it was DevOps the ones that were on call (payed), but they were very picky regarding what they can touch/work on (they almost never touched application code... we should call them "Ops" then, no?).

> Honestly, the whole attitude of not wanting to work more than 8 hours is privilege.

And it's a privlege I'm thankful for. What's wrong with that?

> As a dev, you get a good salary and a job you don't have to break your body to do.

We do break our body to do software engineering (our brains, to be more specifically). If you think physical work >>> brain work, well, that's relative. Every person is different, and for me, brain work is equally taxative as physical work.

> And it helps you as an engineer. Like the article points out, it creates empathy for the users and product support engineers, it helps you improve architecture and app design, and it helps you understand different failure domains

I know I can become better by working harder and smarter (it's obvious), but I just want to be the best version of myself by putting at most 40h/week. Isn't that something honourable in itself? Or does that make me a "bad engineer"?

Re: Why should I do production support?

#132

Earlier quoted context omitted.

This attitude almost always turns into the following: On-call: "Hey devs, I'm being woken up at 3AM because your app sucks. Please fix it." Devs: "Sure, no problem." 4 months go by On-call: "These alerts are still coming in at 3AM. Did you fix the issue?" Dev: "We have a lot of work, we can't dedicate all our time to some minor problems, we have a deadline." Next week, Devs are put on-call. The alerts are fixed in tw…

> This attitude almost always turns into the following: [...] Well, that's another problem (the dev not being able to solve a bug that reappears at 3AM). > The alerts are fixed in two weeks. Site reliability goes up. Apps suddenly become more resilient to failure. I always wondered why DevOps has the "Dev" in its title. At least, in most of the companies I have worked on, it was DevOps the ones that were on call (pay…

DevOps isn't a person or a role. People who are called a 'DevOps Engineer' simply have a bad title. Our industry is very stupid.

But yes, they are often Ops, and in general they shouldn't be troubleshooting the application. They can isolate system issues by looking at and correlating metrics and events in the high level systems, but only a developer of the code in question can efficiently and effectively diagnose a specific bug in code in a reasonable amount of time.

You've seriously not heard the phrase 'check your privilege'? In general it means not to take it for granted, but specifically to not take advantage of it at the expense of others. Just because you aren't forced to care for others doesn't mean you shouldn't. If you have privilege, your moral imperative is to ensure it's used for good, not evil (indifference is often the latter when affected by those with privilege).

How is software dev breaking your body? Are you typing with your face?

If you want to be the best version of yourself within 9-5, please fix the bugs that alert on-call the first morning you hear about them. Most developers never do this, which is why they are put on call.

Last week I was on-call. The vast majority of the time, when it's the application at fault, I don't know who the hell wrote it or where to begin with troubleshooting it, so I need a dev to look at it, because we don't have time to waste if the product is down. This doesn't seem like a controversial idea to me: you are hired by a company to make a product that works, so if your product isn't working, you need to fix it. I can't always fix it. I need help sometimes. It's literally your responsibility as a professional adult to help.

If the product you work on has to work 24 hours a day, you have implicitly agreed to support it during that time. Otherwise you can get a different job where if the product breaks, it can wait until morning. I've had those too, and not being on-call was great! But with my current role I knew it would require some on-call time, because those are the products I'm helping to build and run.

So I spend part of my time improving the product support team so that everything is as resilient as possible, but also so that devs can understand how their code affects the products. This means being as involved in architecture and design decisions as with deploying infrastructure. And I stop at 40hr/week too, but once every 8 weeks, I get a few calls about broken shit, and I put in the time to help prevent those from re-occurring, because most people I work with don't.

Re: Why should I do production support?

#133

I like to write software but I don't want to be on-call if the software I wrote breaks at 3am in the morning. I do take my job with professionalism, I do write tests for most of it (not 100% coverage, but 100% coverage of the critical parts), I do monitoring (and answer and fix alerts if they happen during working hours) and I don't deploy on Fridays (and don't allow people to deploy on Fridays). My code will crash s…

This attitude almost always turns into the following: On-call: "Hey devs, I'm being woken up at 3AM because your app sucks. Please fix it." Devs: "Sure, no problem." 4 months go by On-call: "These alerts are still coming in at 3AM. Did you fix the issue?" Dev: "We have a lot of work, we can't dedicate all our time to some minor problems, we have a deadline." Next week, Devs are put on-call. The alerts are fixed in tw…

This attitude is terrible and exploitative. Employee's have a duty of fidelity and good faith to their employer, sure - but highly skilled and in-demand engineers should demand better of their employees.

Re: Why should I do production support?

#134

You don't get anything out of burning out. You just burn out. And that time is not coming back. This post seems like an apologist talking. Production support alone is not that much of a problem. What the author skipped (conveniently? or forgot to mention?) is - it's really the "on call" phenomenon that's the problem. The "typical" on-call - where when you are on-call you are magically on-call 24x7. Yes, during your s…

> There's no excuse for burning out engineers (esp. unsuspecting youngsters) by pushing them into ungodly hours of work ruining their health among other things while trying to constantly tell them - "do you even realise what a service to humanity you are doing!". In the US, this is happening across the board and not just in tech. The expectation to always be available [often without monetary compensation] is sadly th…

Everyone is doing it is the excuse

Re: Why should I do production support?

#135

Earlier quoted context omitted.

> This attitude almost always turns into the following: [...] Well, that's another problem (the dev not being able to solve a bug that reappears at 3AM). > The alerts are fixed in two weeks. Site reliability goes up. Apps suddenly become more resilient to failure. I always wondered why DevOps has the "Dev" in its title. At least, in most of the companies I have worked on, it was DevOps the ones that were on call (pay…

DevOps isn't a person or a role. People who are called a 'DevOps Engineer' simply have a bad title. Our industry is very stupid. But yes, they are often Ops, and in general they shouldn't be troubleshooting the application. They can isolate system issues by looking at and correlating metrics and events in the high level systems, but only a developer of the code in question can efficiently and effectively diagnose a s…

> How is software dev breaking your body? Are you typing with your face?

I wrote about it. I consider my brain just like any other muscle of my body, so after 8 hours of work, my brain is quite tired. Perhaps my wording was wrong.

> If you want to be the best version of yourself within 9-5, please fix the bugs that alert on-call the first morning you hear about them. Most developers never do this, which is why they are put on call.

Agree.

> you are hired by a company to make a product that works, so if your product isn't working, you need to fix it

Agree as well, but my contract states "40 hours per week". I'm being plain straightforward here, I'm not willing to give anything for free to any company (does that happen the other way around? Never). Not sure what's "wrong" with this.

> And I stop at 40hr/week too, but once every 8 weeks, I get a few calls about broken shit, and I put in the time to help prevent those from re-occurring, because most people I work with don't.

I have nothing against that, and I respect it. I guess the other way around should work as well, right? Like, if someone is not willing to give more than what's stated in their contract, that should be fine for everyone. What you call "help" sure it's help, but companies are taking it as free labor. I have nothing against companies making money, but I do not support companies making money without having to pay employees for that. But the main point of my first comment was: even though companies are paying for being on-call, one should have the option to say 'No, thanks. I don't want to give you my free time in exchange for more money. I already have enough with my 40h/week schedule', and that should be fine for everybody.

Re: Why should I do production support?

#136

Earlier quoted context omitted.

> This attitude almost always turns into the following: [...] Well, that's another problem (the dev not being able to solve a bug that reappears at 3AM). > The alerts are fixed in two weeks. Site reliability goes up. Apps suddenly become more resilient to failure. I always wondered why DevOps has the "Dev" in its title. At least, in most of the companies I have worked on, it was DevOps the ones that were on call (pay…

DevOps isn't a person or a role. People who are called a 'DevOps Engineer' simply have a bad title. Our industry is very stupid. But yes, they are often Ops, and in general they shouldn't be troubleshooting the application. They can isolate system issues by looking at and correlating metrics and events in the high level systems, but only a developer of the code in question can efficiently and effectively diagnose a s…

> How is software dev breaking your body? Are you typing with your face?

Desk work comes with significant bodily stresses which ergonomics can only partially ameliorate.

I'll take it over digging ditches, sure. But I won't do more than six hours of it a day, that's as long as I can healthily stand, and several hours longer than I can sit without pain.

Post reply on HN