Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: I salute everyone on call/working support through the holidays

news.ycombinator.com

Post reply on HN
Print view View original

181–190 of 207 posts

Re: Tell HN: I salute everyone on call/working support through the holidays

#181

Post by crossroadsguy »

tdeck wrote: post #151
crossroadsguy wrote: post #106

Earlier quoted context omitted.

And I am not even sure whether you are talking about just day-time on-call or the 24 hours on-call for at least 1 full week to two week stretches or a simple 12 hours on-call you are talking about? In India the Indian managers (and American managers are just fine with it) have made an environment of this barbaric practice of 24x7 on-call handled by just one person. In fact, even when there are US/western counterparts…

> have made an environment of this barbaric practice of 24x7 on-call handled by just one person. If it makes you feel any better this is very common in small to mid-sized US tech companies as well. In every team I've been on that had an oncall rotation it was a full week 24/7 per person, that rotated among team members. Even at Google we were on call for our own service overnight and didn't have SRE / other time zone…

Why would that made me feel better, tdeck? It doesn't make me feel better.

Besides it’s different what you mentioned about where you are from and what I experience and see as the norm where I am from.

Re: Tell HN: I salute everyone on call/working support through the holidays

#182

Post by jle17 »

JoshTriplett wrote: post #87
chunkymilk wrote: post #34

Earlier quoted context omitted.

> Always be kind, and say it’s your fault. I do this with internal teams at work. I've found approaching other teams with issues with their library/framework in a "this could be our mistake" manner really helps in keeping them from getting defensive and stonewalling.

Absolutely. The same is true for bug reports: if you always approach any bug report with the possibility that it may be your mistake, 1) you avoid annoying someone if it is your mistake, and might actually get helpful advice, and 2) you're more likely to get a cordial reception for real bugs. You don't have to be excessively self-effacing about it, just avoid presenting things as though the project you're reporting i…

It's also important to consider how what you view as a bug might not be one from the point of view of the person treating your issue. It is so, so infuriating to receive a "bug" report asking you to "fix" something that is in fact a feature request for something that is not implemented yet.

Even if you get an error from the software, consider that you might not be using it as intended or setup properly.

Re: Tell HN: I salute everyone on call/working support through the holidays

#183

Post by _kb »

wavesbelow wrote: post #180
_kb wrote: post #176

Earlier quoted context omitted.

I completely acknowledge it's utopian but isn't it a better goal to target continuous stability, or at least semi trusted process for when things inevitably break? It's a similar concept to not deploying on Fridays. If you're afraid to introduce changes due to some arbitrary timing, perhaps it's worth focusing on the source of that uncertainty.

It's not either/or. The observation that freezes and no Friday deployments capitalize on is that the single most likely cause of production incidents is production changes. We always should target better stability, but no matter how good your system and incident response are, if your goal is to minimize customer disruption during a certain time window, or avoid dealing with incidents on weekends, minimizing productio…

I agree. The flipside though is a freeze also invokes a scenario that can lead to a premature release. Any blanking window forces a decision between deferring or rushing work, neither of which are ideal.

Re: Tell HN: I salute everyone on call/working support through the holidays

#184

Post by bertil »

ZeroSolstice wrote: post #161
bertil wrote: post #25

Always be kind, and say it’s your fault. If you don’t do it for the sake of the person you are asking for help, do it because it works better. That’s the most practical advice [0] ever given by Hans Rosling [1], the Fact master himself: > In fact, I have the secret to how to get the best help immediately from any customer service, like the phone company or the bank or anything. I have the best line, it always works.…

I'm going to be a bit pedantic and say fault and mistake are not equal in the message they convey. Hans says he has 'made a mistake' which is not the same as saying 'I'm at fault.' In the end it might be proven that you are at fault but until then all you know is that something went wrong, isn't working, etc.

I think that’s the kind of pedantry that is relevant in that case and in designing your product and customer support tree.

Friends of mine hearing him would say, He never says what the mistake is precisely, but there’s always the option that it was booking a flight with that airline.

Re: Tell HN: I salute everyone on call/working support through the holidays

#185

Post by makeitdouble »

chunkymilk wrote: post #34
bertil wrote: post #25

Always be kind, and say it’s your fault. If you don’t do it for the sake of the person you are asking for help, do it because it works better. That’s the most practical advice [0] ever given by Hans Rosling [1], the Fact master himself: > In fact, I have the secret to how to get the best help immediately from any customer service, like the phone company or the bank or anything. I have the best line, it always works.…

> Always be kind, and say it’s your fault. I do this with internal teams at work. I've found approaching other teams with issues with their library/framework in a "this could be our mistake" manner really helps in keeping them from getting defensive and stonewalling.

You'd probably get the same response approaching them with a "we're trying to do this but couldn't make it work, could you help us ?" kind of pitch.

Except if you're actually convinced it could be your mistake, getting that tone will feel like getting played like some small kid. Most people will help you anyway and be professional, of course.

That reminds me of detective Columbo. It's cute and all when it's supposed to be done to strangers. Imagine Columbo coming to you every week with that same convoluted spiel.

Re: Tell HN: I salute everyone on call/working support through the holidays

#186

Post by dkjaudyeqooe »

tdeck wrote: post #152
dkjaudyeqooe wrote: post #127

Earlier quoted context omitted.

With the last (and only) job that required me to be on call I quit the day before I was scheduled. I've always refused to do it. Devs have no business doing it.

I appreciate setting boundaries but I don't really understand this attitude. Frequently on call issues are caused by problems with the application logic, therefore solving them requires an understanding of the code. It's not usually my experience that oncall issues are a simple case of force-restarting something or provisioning more boxes, although that can happen from time to time.

A system that can get itself into a non-functioning state and that can't be supported by an operator or dedicated support person is fundamentally broken and should not be in production. In my view devs should never have access to production, under any circumstances, ever.

This is an artifact of devs (and others) not knowing what they're doing, and just hacking and hoping for the best. It's really not that hard to develop a system that is reliable and supportable in a basic way. Understanding the code shouldn't be a requirement, but understanding the system should, and that's a requirement of support personnel. Put another way, the functional model of the system has to be at a higher level than the code.

Re: Tell HN: I salute everyone on call/working support through the holidays

#187

Post by ciceryadam »

ponector wrote: post #73

Is it a US thing to push updates right before holidays and force people to be on call? European companies I've been working on are planing releases till the end of November, first week of December max. All project plans are baked like there is one week in December. While US company tried to force every contractors to work everyday and some weekends too. Are you Indian? You don't celebrate Christmas, you'll be on call…

SRE teams are Oncall 24/7/365

Re: Tell HN: I salute everyone on call/working support through the holidays

#188

Post by [deleted] »

JoshTriplett wrote: post #88
ponector wrote: post #73

Is it a US thing to push updates right before holidays and force people to be on call? European companies I've been working on are planing releases till the end of November, first week of December max. All project plans are baked like there is one week in December. While US company tried to force every contractors to work everyday and some weekends too. Are you Indian? You don't celebrate Christmas, you'll be on call…

Not all breakage happens because of new updates. Sometimes the servers go down, sometimes a bug that already existed gets triggered by something, sometimes there's extra volume over the holidays and the load is too much. Full agreement that you shouldn't push new code in late December, but that won't stop the need for some folks to be on call.

[deleted]

Re: Tell HN: I salute everyone on call/working support through the holidays

#189

Post by datadrivenangel »

dayjaby wrote: post #164
jeffhuys wrote: post #146

Earlier quoted context omitted.

Do NOT do this too much, though. I did this in my last relationship and it turned into resentment from me, while she walked all over me, knowing I would say sorry for everything. It wasn’t her fault, I should’ve protected my values more, but this is a slippery slope. However, my next partner in my opinion should take this and say something like “no, this is not your fault. Let’s work on it”. It could go both ways. I…

Admitting weakness gets exploited very easily by people. Me: Sorry, I forgot about some details about the stuff I was working on last month. Him: Your memory is so bad. I can literally remember all the details of the last 6 months. Try to become more like me.

And this is a great asshole filter.

Re: Tell HN: I salute everyone on call/working support through the holidays

#190

Post by Octabrain »

dkjaudyeqooe wrote: post #127
Octabrain wrote: post #95

On call sucks so badly. At this point of my life, I firmly believe that there's not enough amount of money that can compensate the mental suffering it implies. Even more if the company you work for has this mentality of "deal with it" without making improvements, which was my case in the last period I did on call and what made the camel's back to break for me. Nowadays I simply refuse it. For those who are still on t…

With the last (and only) job that required me to be on call I quit the day before I was scheduled. I've always refused to do it. Devs have no business doing it.

> Devs have no business doing it.

Agree but, I have to say that, as a DevOps, it was infuriating to me to have to deal with developers without any care for the quality of what they were delivering. Sometimes for pressure from someone higher in the chain, other times, for pure laziness and/or incompetence. I remember coming in the morning after a hell of a night on the on-call, reporting the issues to the Devs in charge and being answered something along the lines of "fixing that is not the priority right now" and my replying on anger with "If it was your damn phone the one ringing during the whole night I'm pretty sure you would make it a priority".

Post reply on HN