If you're on a software team that maintains cloud services or other infrastructure that needs to keep running 24x7, how does your team decide who is going to work oncall for shifts like Thanksgiving, Christmas, New Year's, or other times when most people don't want to be oncall?
Is it just a top-down manual manager decision? Or do you have some kind of process/algorithm to help make it fair? Do new people get worse oncall shifts than people who have been on the team for longer?
Ask HN: How does your team decide who works oncall on Christmas?
1–10 of 15 posts
Re: Ask HN: How does your team decide who works oncall on Christmas?
#2When I had a team, we'd trade based on faith -- Jewish or Muslim folks worked Christmas, and on their holidays us "cultural Catholics"[1] covered so they could have "their" day off.
Even among those who were no longer religious, folks often had families who still hold these beliefs.
(I was educated in a Catholic school, was an altar boy, and will never return to either place, so I use the term "cultural Catholicism" to distinguish my agnosticism from the full blown atheism I see on HN which often has a bit too right libertarian branding for my left libertarian tastes.)
Re: Ask HN: How does your team decide who works oncall on Christmas?
#3Well back in the day it wasn't a problem because we had an on call allowance and you got paid for that, nowadays they told everyone
"you're too good for hourly work, we're going to make you salaried, it's great now you can just focus on getting your work done and not have to worry about hours on the clock. But you still have a timesheet and we still expect you to be here 9-5 and you can't leave early unless it's approved by your manager. Also you'll have to work 60-80hrs a week until we are caught up which should be sometime around Q3 of next year. But you're empowered now and we are flexibile"
Re: Ask HN: How does your team decide who works oncall on Christmas?
#4When I had a team, we'd trade based on faith -- Jewish or Muslim folks worked Christmas, and on their holidays us "cultural Catholics"[1] covered so they could have "their" day off. Even among those who were no longer religious, folks often had families who still hold these beliefs. (I was educated in a Catholic school, was an altar boy, and will never return to either place, so I use the term "cultural Catholicism"…
Didn't you still run into times when most people wanted the time off? For example, times when schools are off, so everybody who is a parent wants that time off, regardless of whether it lines up with their cultural or religious important dates?
Re: Ask HN: How does your team decide who works oncall on Christmas?
#5Maybe the better question is why is this a problem? I've been in oncall rotations my entire multi decade career and in the beginning I was frustrated that I was getting called all the time. Now I understand that the reason for that was bad management/owners.
I'm still oncall but if there's a issue that triggers a review to make changes so it never happens again. N+1 on everything, and N+2 redundancy on critical systems. A single failure should be a non-issue. It should take multiple failures inside of a single system to engage the oncall. Human error should be eliminated by not allowing changes during big holidays. Many companies lock down from Thanksgiving to new years.
While things can and will still happen, properly implemented and managed systems should eliminate the most if the oncall issues. The ones that do come up will be an "all hands on deck" kind of thing. People will be less afraid of oncall then they know they rarely get called.
Re: Ask HN: How does your team decide who works oncall on Christmas?
#6When I had a team, we'd trade based on faith -- Jewish or Muslim folks worked Christmas, and on their holidays us "cultural Catholics"[1] covered so they could have "their" day off. Even among those who were no longer religious, folks often had families who still hold these beliefs. (I was educated in a Catholic school, was an altar boy, and will never return to either place, so I use the term "cultural Catholicism"…
Didn't you still run into times when most people wanted the time off? For example, times when schools are off, so everybody who is a parent wants that time off, regardless of whether it lines up with their cultural or religious important dates?
No
Re: Ask HN: How does your team decide who works oncall on Christmas?
#7Well back in the day it wasn't a problem because we had an on call allowance and you got paid for that, nowadays they told everyone "you're too good for hourly work, we're going to make you salaried, it's great now you can just focus on getting your work done and not have to worry about hours on the clock. But you still have a timesheet and we still expect you to be here 9-5 and you can't leave early unless it's appr…
They used to give whoever was on call for the week $200, we had a guy saving up for his wedding so he offered to be on-call for anyone who didn't want to be on call. It was a good deal for him and everybody was happy. It's amazing how cheap companies are today, for $10K you could make a lot of people happy.
Re: Ask HN: How does your team decide who works oncall on Christmas?
#8Maybe the better question is why is this a problem? I've been in oncall rotations my entire multi decade career and in the beginning I was frustrated that I was getting called all the time. Now I understand that the reason for that was bad management/owners. I'm still oncall but if there's a issue that triggers a review to make changes so it never happens again. N+1 on everything, and N+2 redundancy on critical syste…
I think what you describe is the ambition of every oncall team. But even when this works really well, so the calls are few and far between, just being scheduled to work oncall means you're more limited in what you can do. You've got to be reachable by phone. You have to be within a few minutes of accessing a computer with good networking. There'll be no road trip drives or flights to see the in-laws while you're the one on oncall duty.
Do you think people prefer to avoid just being on oncall duty over big holidays, even if there aren't many calls? Perhaps that preference only shows up in a small portion of oncall teams.
Re: Ask HN: How does your team decide who works oncall on Christmas?
#9Generally they will hire foriegn workers on H1b.
Re: Ask HN: How does your team decide who works oncall on Christmas?
#10Just give people extra off-days, like 2 days per each 1 of oncall for those days, or just give them more Oncall money. If there is the incentive and it is good, people will step up to do it, and overall that shouldn't be costly for a company that needs to run their infra 24x7.