Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why should I do production support?

devenbhooshan.wordpress.com

Post reply on HN
Print view View original

61–70 of 136 posts

Re: Why should I do production support?

#61

Post by zeckalpha »

crossroadsguy wrote: post #9

You don't get anything out of burning out. You just burn out. And that time is not coming back. This post seems like an apologist talking. Production support alone is not that much of a problem. What the author skipped (conveniently? or forgot to mention?) is - it's really the "on call" phenomenon that's the problem. The "typical" on-call - where when you are on-call you are magically on-call 24x7. Yes, during your s…

If the engineers aren’t oncall, who is? Is it okay to exploit non-engineers? If anything, it is less exploitative to have those who are empowered to improve their situation oncall.

Re: Why should I do production support?

#62

Post by detaro »

zeckalpha wrote: post #61
crossroadsguy wrote: post #9

You don't get anything out of burning out. You just burn out. And that time is not coming back. This post seems like an apologist talking. Production support alone is not that much of a problem. What the author skipped (conveniently? or forgot to mention?) is - it's really the "on call" phenomenon that's the problem. The "typical" on-call - where when you are on-call you are magically on-call 24x7. Yes, during your s…

If the engineers aren’t oncall, who is? Is it okay to exploit non-engineers? If anything, it is less exploitative to have those who are empowered to improve their situation oncall.

Given that the parent comment is clear about the problem being the 24x7 expectation: Someone paid to work or be available that shift, engineer or not?

Re: Why should I do production support?

#63

Post by [deleted] »

zeckalpha wrote: post #61
crossroadsguy wrote: post #9

You don't get anything out of burning out. You just burn out. And that time is not coming back. This post seems like an apologist talking. Production support alone is not that much of a problem. What the author skipped (conveniently? or forgot to mention?) is - it's really the "on call" phenomenon that's the problem. The "typical" on-call - where when you are on-call you are magically on-call 24x7. Yes, during your s…

If the engineers aren’t oncall, who is? Is it okay to exploit non-engineers? If anything, it is less exploitative to have those who are empowered to improve their situation oncall.

[deleted]

Re: Why should I do production support?

#64

Post by tybit »

afberg wrote: post #60
crossroadsguy wrote: post #9

You don't get anything out of burning out. You just burn out. And that time is not coming back. This post seems like an apologist talking. Production support alone is not that much of a problem. What the author skipped (conveniently? or forgot to mention?) is - it's really the "on call" phenomenon that's the problem. The "typical" on-call - where when you are on-call you are magically on-call 24x7. Yes, during your s…

I think there's only one way to solve this — which I've been unsuccessfully advocating for at my current company — and that is paid voluntary on call schedules. It creates an actual market for on call work where engineers can simply say no to the extra cash if they don't like work taking up their nights and weekends. If the company is having trouble with no engineers wanting to be on call the pay is simply too low an…

I work somewhere that does this and it works surprisingly well. It’s rare to find an employer that’s willing to look after their staff though.

Re: Why should I do production support?

#65

Post by sdevonoes »

I like to write software but I don't want to be on-call if the software I wrote breaks at 3am in the morning. I do take my job with professionalism, I do write tests for most of it (not 100% coverage, but 100% coverage of the critical parts), I do monitoring (and answer and fix alerts if they happen during working hours) and I don't deploy on Fridays (and don't allow people to deploy on Fridays).

My code will crash sooner or later. I already know that. I don't write 100% bug-free code. But I cannot accept to give 100% of my time one week per month or so to a company in exchange for money. I just don't understand why people can't understand that I can be a professional only during 8 hours per day, but not more.

Re: Why should I do production support?

#66

Post by hermitcrab »

I am an independent developer living off 3 software products I created and sell. I have done all my own support over the last 15 years. While it can be frustrating (especially for B2C) it is also my superpower, as it gives me some much more insight into how I can improve my products. I only do support via email (not phone or chat).

Re: Why should I do production support?

#67

Post by cgrealy »

ocdtrekkie wrote: post #7

If you aren't doing production support, you don't actually know your product. You aren't connected to the pain points your users experience and you miss what is, to your support team and your customers, the glaringly obvious. I would argue all developers should be required to do some support work.

Sorry, this is a terrible argument.

No one expects customer support people to write code. Why? Because they don't have the skillset.

Yet people who make this argument seem to think any moron can do support.

The skillset for an engineer is not a superset of a customer support person.

Have your engineers sit in on support, by all means, but actually making them DO support will result in unhappy engineers and sub-par support.

Do not undervalue a good support person. They have a whole suite of skills engineers often don't have.

Re: Why should I do production support?

#68

Post by piokoch »

nineteen999 wrote: post #37
dopylitty wrote: post #17

Earlier quoted context omitted.

The point about on-call is really critical and really on-point. If a company thinks an application is important enough to run 24x7 then it should staff for 24x7 support. Stealing wages from workers by expecting them to be available 24x7 (on-call) is an absolute abuse. It also leads to burn out, poor performance during the day (how is a dev's development ability when they were up at 2:30am on an incident call the nigh…

> If a company thinks an application is important enough to run 24x7 then it should staff for 24x7 support And where it really matters, they do. My team and I build and manage a large Emergency Services telecommunication network. We have Tier 1/2 operators on shift work 24/7. Tier 3 staff (programmers, system integrators and administrators) are their escalation point for critical issues outside of business hours. > S…

"Frequently they don't get called during their week at all and it's free money."

That's, I think, is a wrong perspective, when people are on call, they have to be somewhere near their computer/internet connection and be ready to work (so it is not just you can go to a party and if call happens do some quick fix in a toilet).

On-calls cannot do what they want with their time, so they don't get money for free.

Re: Why should I do production support?

#69

Post by jake_morrison »

A lot of the pain around production support is easily solved by having staff in multiple time zones.

A good structure is to have first line support be relatively generic ops people. They can handle problems related to infrastructure, e.g. hardware failures, network problems, or issues that can be handled by adding resources. The deployment process should be consistent enough across applications that they can e.g. roll back to a previous release.

This covers the majority of production problems. After that, it's time to bring in someone who understands the details of how the application works. If the dev team is geographically distributed, then someone is available during working hours. Otherwise, we have to get someone out of bed.

If the dev team has done their job right, this should be a rare occasion. Making the dev team fully responsible for the reliability of the application means that they are motivated to make it reliable. Otherwise there is a tendency to have an underclass of ops people who get abused.

A fundamental mindset here is taking responsibility for the user experience, including reliability. If this is not owned by the product development team, then who?

Re: Why should I do production support?

#70

Post by imhoguy »

hinkley wrote: post #54
octodog wrote: post #33

Earlier quoted context omitted.

1. Not every support request is the result of a bug. 2. Not every bug is directly related to the product owner/team. Bugs can be introduced from different teams/processes. 3. Requiring senior engineers to do L1 and L2 support is likely a misallocstion of resources when less senior people can handle those issues.

4. Engineers have a bad habit of renegotiating scope in front of the customer without realizing they’ve done it.

As usual, "that should be easy to fix, just line of code here". At least from SWE perspective the fix may be easy to make but architecture or product management team may not be so happy about such pitch, especially when customer responds with "oh great, when?".
Post reply on HN