Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Oncall shift should be Tuesday to Tuesday

arthur-johnston.com

Post reply on HN
Print view View original

151–160 of 230 posts

Re: Oncall shift should be Tuesday to Tuesday

#151

Post by 8note »

dangus wrote: post #81
sokoloff wrote: post #78

Earlier quoted context omitted.

As long as the on-call coverage is as specified at the time of hiring, this is just a difference in form of payment . If I receive 100 total units of compensation, I'd way rather get 100 units of base pay (and 0 on-call pay) than 90 units of base pay and 10 units of specific on-call pay. (What if the company eliminates on-call? What if I get injured and my insurance only covers base pay? Severance is usually based on…

How can the on-call coverage be specified at hiring? Can the company guarantee that my team will never shrink or that the page rate won't increase? What will financially encourage my company to stop paging me overnight if there isn't a labor cost to the company every time an on-call incident occurs? > What if I get injured and my insurance only covers base pay? Insurance payouts can be easily based on wages that incl…

> PTO days and on-call days don't indersect.

If you have any national holidays, somebody still ends up being on-call for that holiday. I've been on-call for almost every US holiday this year.

Re: Oncall shift should be Tuesday to Tuesday

#152

Post by QuarterReptile »

0xbadcafebee wrote: post #124
sed_zeppelin wrote: post #104

I just want to jump in as a minority voice here. In case anybody is reading the other comments and feeling... alienated. I refuse to accept on-call duties, full stop. If a job posting expects it, I don't apply. If a hiring manager says they have it, I do not accept the offer. If management starts talking about maybe implementing it, I protest. If it becomes enacted, I resign. There is absolutely no situation in which…

I am 40 yrs old. I get paid a shit-ton of money (just around $200K) to do this stupid tech work job. I work 40 hours a week, I get benefits, flex time, plus I work remote. If I'm getting paged for a legitimate issue that is related to something I built or maintain, then, yes, I am going to respond on-call. Because it's a fucking privilege to get paid this much money to sit on my ass and type into a screen. If I'm get…

You haven't lived until you've spent a whole weekend at work rushing to fix a production-limiting issue because the boss doesn't know, though you do, about the other division's production-limiting issue which cannot, under any wildly optimistic circumstance, get done in the next two weeks.

Oh, and that weekend is the weekend before Christmas.

Re: Oncall shift should be Tuesday to Tuesday

#153

Post by quicklime »

mikeocool wrote: post #146
ggeorgovassilis wrote: post #141

Earlier quoted context omitted.

> You expect to not be responsible for what happens to the software you put into production? First: IT seems to be rather the exception - most professions have no on-call. Eg. even if my car mechanic screws up a service job, they'll have me bring the car back into the garage during their normal working hours, regardless of how and where stranded I am in the middle of the night. A second comment: I'll be responsible f…

> IT seems to be rather the exception I think there’s actually a fair number of jobs where some level of this is expected. Doctors are one obvious example — they have on call responsibilities often more onerous than IT, and depending on the situation don’t always receive additional compensation for it. If you manage people who work different hours from you, in a lot of jobs it’s not uncommon to be called in if shit h…

> Doctors are one obvious example — they have on call responsibilities often more onerous than IT, and depending on the situation don’t always receive additional compensation for it.

My doctor (primary care physician) doesn’t work outside of business hours. In an emergency the recorded message says to call an ambulance and go to the emergency department at the hospital, which is staffed by a different set of people.

So it seems they do have at least some separation of the oncall aspect?

Lawyers are another story, there’s a lot of things wrong with that profession and we shouldn’t be trying to copy them.

Re: Oncall shift should be Tuesday to Tuesday

#154

Post by phamilton4 »

When did on-call become so accepted and demanded from employers? Currently I am "Release Captain" for a week: So I have to setup any releases and manage all the related tasks, do automated/manual testing of the release, release (enabling toggles and any config changes). Then Backup to secondary and primary for a week: About once or twice I am asked to help with tickets. Then for 14 days we alternate primary / secondary. Thursday to Thursday is our deal. Every ~40 days I am in one of the above. It's absolutely miserable.

I have never had this much time spent doing non-development related tasks. For 4 weeks every 1.5 months I can't have a life at all. This just screams to me that we are forcing broken software/not complete software out the gate a building huge piles of technical debt that will never get the focus. I remember a time when I would start at 9am and end at 6pm every day and never heard a peep about production issues unless the support engineers couldn't figure it out. Which maybe happened twice a year. To make matters worse most things are not allowed to be touched in production with the risk of being fired for making changes. So if you want to "fix" any data or call xyz service you need high ranking approval. It's like being tortured!

Re: Oncall shift should be Tuesday to Tuesday

#155

Post by joshuamorton »

Retric wrote: post #145
deathanatos wrote: post #125

Earlier quoted context omitted.

You expect to not be responsible for what happens to the software you put into production? (… and I'd like to avoid distracting arguments that amount to "my company does on-call badly" — yeah, those problems do exist and we should strive to fix them. But if I'm to not categorize the argument here as the baby with the bathwater, then we need something to replace on-call with. Prod goes down on a Saturday afternoon; ar…

I don’t put things in production, the company does. And it’s the companies responsible to deal with problems that show up. 24/7 coverage is expensive and mandating someone is on call 24/7 don’t actually provide it.

This is "companies do on-call badly".

For the purposes of this exercise presume that our theoretical on-call process is no worse than Google's SRE structure: You are on-call for a 12 hour shift that is more or less aligned with your waking hours, and you are compensated extra for the time you are on-call outside of normal working hours, whether or not you are called in. You are on-call at most one week per month, on average, and usually less.

Re: Oncall shift should be Tuesday to Tuesday

#156

Post by dheera »

sed_zeppelin wrote: post #104

I just want to jump in as a minority voice here. In case anybody is reading the other comments and feeling... alienated. I refuse to accept on-call duties, full stop. If a job posting expects it, I don't apply. If a hiring manager says they have it, I do not accept the offer. If management starts talking about maybe implementing it, I protest. If it becomes enacted, I resign. There is absolutely no situation in which…

I 100% fully agree with you.

I have survived 2 cardiac arrests (almost died) during high-stress times. I've been stable for a few years now, but only after I enacted VERY HARD boundaries around work/life and never cut down on sleep for any reason (among other health-first changes I made). I have a significant increase in cardiac arrythmias any time I don't sleep enough.

I consider myself at this point as having a disability that prevents me from overworking, and I absolutely need my employers to respect that and accommodate that.

I can work normal hours, and that's my offer. If you want to pay me less, that's okay, but I'm not doing on-call unless it's business hours only.

If customers give a shit about uptime at 2am then it's management's responsibility to find people in other time zones to deal with it, or pay extra for people who are willing to sacrifice and risk their health for a customer (I won't take that deal though).

Re: Oncall shift should be Tuesday to Tuesday

#157

Post by nosefurhairdo »

sed_zeppelin wrote: post #104

I just want to jump in as a minority voice here. In case anybody is reading the other comments and feeling... alienated. I refuse to accept on-call duties, full stop. If a job posting expects it, I don't apply. If a hiring manager says they have it, I do not accept the offer. If management starts talking about maybe implementing it, I protest. If it becomes enacted, I resign. There is absolutely no situation in which…

I've been the on call engineer on my team for 75+% of the last year (most of my team is contractors, new hire not onboarded to on call rotation yet, etc.).

It's not an issue because we don't break prod. I also feel I'm well compensated. When there have been issues at inconvenient hours, my manager has encouraged me to take it easy after resolving the incident. We've also prioritized improving our integration tests and addressing other issues noted during root cause analysis (RCA), which I suspect is why we haven't had any incidents in recent memory.

If on call duties are this frustrating, I'd argue it's team/organizational dysfunction that is the real problem, and bad on call shifts is just one of the symptoms.

Ultimately, somebody needs to be available to fix a production incident. One person suffering from on call duties is better than thousands of paying customers suffering from broken software.

Re: Oncall shift should be Tuesday to Tuesday

#158

Post by dheera »

deathanatos wrote: post #125
sed_zeppelin wrote: post #104

I just want to jump in as a minority voice here. In case anybody is reading the other comments and feeling... alienated. I refuse to accept on-call duties, full stop. If a job posting expects it, I don't apply. If a hiring manager says they have it, I do not accept the offer. If management starts talking about maybe implementing it, I protest. If it becomes enacted, I resign. There is absolutely no situation in which…

You expect to not be responsible for what happens to the software you put into production? (… and I'd like to avoid distracting arguments that amount to "my company does on-call badly" — yeah, those problems do exist and we should strive to fix them. But if I'm to not categorize the argument here as the baby with the bathwater, then we need something to replace on-call with. Prod goes down on a Saturday afternoon; ar…

I am capable of writing very good software, testing it, and putting it into production, but I am not capable of being responsible for what happens at 3am on a Sunday. Whether that deal is okay is up to you. I'm okay if you don't want to pick me. There are other jobs I can get. I write good software though.

If the customer is awake at 3am on a Sunday, it's the customer's problem that they were awake at 3am on a Sunday. If it's a social network, I frankly couldn't care; the customer should go to bed. If it's going to be deployed in the emergency room, fine, we should care, but YOU, management, should find people who are actually willing to take that shift (for extra money, or are based in other time zones).

Re: Oncall shift should be Tuesday to Tuesday

#159

Post by nosefurhairdo »

mikedelfino wrote: post #150
hn_go_brrrrr wrote: post #135

Earlier quoted context omitted.

"you need 5-6 people" is moving the goalposts. The root comment said nothing about minimum team size.

If the company has enough people in the team, someone just works the night shifts or on scheduled weekends. No one needs to be on-call because there would be someone taking care of it already.

Is the argument here that every software team should have engineers whose normal working hours have 24/7/365 coverage?

Re: Oncall shift should be Tuesday to Tuesday

#160

Post by rr808 »

phamilton4 wrote: post #154

When did on-call become so accepted and demanded from employers? Currently I am "Release Captain" for a week: So I have to setup any releases and manage all the related tasks, do automated/manual testing of the release, release (enabling toggles and any config changes). Then Backup to secondary and primary for a week: About once or twice I am asked to help with tickets. Then for 14 days we alternate primary / seconda…

> When did on-call become so accepted and demanded from employers?

As a 50 something year old software engineer. Its always been like this. I'm kinda shocked at how reluctant the new generation is to support the systems. Sure we'd all prefer strict 9-5 hours but most companies rely on software to stay in business and you need experts available in case things go wrong.

Post reply on HN