Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Being on call sucks

bobbiechen.com

Post reply on HN
Print view View original

251–260 of 282 posts

Re: Being on call sucks

#251

Post by hyperman1 »

tharkun__ wrote: post #248
hyperman1 wrote: post #211

Earlier quoted context omitted.

I tend to agree with you, but found an exception a while ago. A certain file has to appear before a specific time, or else some people don't get money they deserve and rightfully get very angry. Except 1 or 2 times per year there is nobody in that situation. No payments have to be made. No file appears, as other alerts would signal an empty file. As the relevant time was in business hours and the thing was important,…

Unacceptable. Make sure that the file is there but empty if nobody needs to get money. If empty could be a failure case, have a 'this page intentionally left blank' type arrangement for the file contents. Done, no monitor exceptions.

Sorry, been there done that. Layout of the file is dictated by an external party. Empty or dummy file is major bad news that blocks other payments. If the file exists, it should have at least 1 valid payment. Payment of 1 cent to a dummy account is illegal. File is produced and consumed by 2 different software packages from 2 different vendors. If the business knowingly creates an invalid file, they commit fraud.

To be honest, calling a human and asking if they are really sure in this case is probably a good idea.

Re: Being on call sucks

#252

Post by Sohcahtoa82 »

muttled wrote: post #165
Broken_Hippo wrote: post #33

Earlier quoted context omitted.

While I agree totally that you Shouldn't be, we all know that some places will try to force this on you.

I had an awful manager who insisted I needed to "see if I can find someone to cover" for me for a funeral.

I'd tell them that's their job as a manager.

Re: Being on call sucks

#253

Post by mlyle »

reddiky wrote: post #249
mlyle wrote: post #179

Earlier quoted context omitted.

> What do you mean by "expectation"? One day on call, one day paid overtime; anything else is abusive. I don't think being available for work if an unlikely emergency comes up usually justifies the same comp as actively working that entire time.

If you're having to actively shape your real life to accommodate your work life then you should be compensated (Not attending events / bringing your computer with you to dinner, etc). Do firefighters only get paid for the amount of time actively fighting fires?

Again, I didn't say that you shouldn't be compensated-- just that you shouldn't be compensated at the rate if they were having you actively doing work the entire time (else, why not just have you actively do work the whole time?).

> Do firefighters only get paid for the amount of time actively fighting fires?

I believe firefighters tend to do 24 hour shifts, where they actively work 8 hours and are paid full rate for that (during which time they have responsibilities in maintenance, etc)...

...and are paid a reduced rate for the other 16 hours they are on standby. (Here, in my jurisdiction, it's 10 hours + 14 hours). During this time they hang out at the firehouse, but may be eating, watching TV, sleeping, etc.

They do 3 of these 24 hour periods per week.

Re: Being on call sucks

#254

Post by mlyle »

sam0x17 wrote: post #243
mlyle wrote: post #192

Earlier quoted context omitted.

DoD comps at 10% of overtime rate when not actively working an issue. https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/14... The comment I replied to appeared to advocate for 100%.

I would advocate for 100% but 10% is way better than 0%. I didn't have any on-call peers during my time in DoD so I'm unfamiliar with the structure for that arrangement, but very familiar with regular overtime stuff with DoD.

10% of time-and-a-half (15% of normal) seems reasonable for time that you are expected to be able to do your own thing but stay available. I negotiated 25% of normal pay for similar things.

24 hours of call at 15%, where they never call you, becomes a half day of pay. If you get a call, whether brief or using the whole 2 hour chunk, you're basically paid for a full normal day. This all feels just and reasonable to both sides.

100% of overtime rate doesn't make sense to me because then they're paying just as much as they'd pay to have you actively work the entire time.

Re: Being on call sucks

#255

Post by Tainnor »

ricardobayes wrote: post #76

After being a SWE for a decade in Europe, I have never heard of anyone in my network who needed to be on-call. Is this a US thing or only for devops? Why would a software engineer need to be on call ever? That just means the CICD/testing/validation pipeline sucks.

I'm sure there's companies in Europe that have on-call shifts, but I've had several jobs now and the only company that ever asked my team to do on-call outside of regular working hours was a US company and we had to explain to them how German labour laws work (specifically that you have to pay people for being on-call and that there are legally required rest periods).

Re: Being on call sucks

#256

Post by tharkun__ »

hyperman1 wrote: post #251
tharkun__ wrote: post #248

Earlier quoted context omitted.

Unacceptable. Make sure that the file is there but empty if nobody needs to get money. If empty could be a failure case, have a 'this page intentionally left blank' type arrangement for the file contents. Done, no monitor exceptions.

Sorry, been there done that. Layout of the file is dictated by an external party. Empty or dummy file is major bad news that blocks other payments. If the file exists, it should have at least 1 valid payment. Payment of 1 cent to a dummy account is illegal. File is produced and consumed by 2 different software packages from 2 different vendors. If the business knowingly creates an invalid file, they commit fraud. To…

I never said 1 cent payment. I didn't know diddly squat about what your domain was or any specifics. You just said that that was a good case of 'bad monitoring is OK'. It's not. Maybe your hands are tied, I'll give you that. It won't make it good though.

Also I said 'this space intentionally left blank'. Whatever actual form that would take in your example. Just because multiple entities use the same interface does not mean a bad interface is suddenly a good interface.

To take an analogy and stay in payments (not sure what your specifics are). If your live system for CC processing OKs a payment from 4111 1111 1111 1111 you're toast. Your system better only 'process' that in a test environment. Perfect for a dummy row that gets ignored but ensures your monitoring does not trigger.

And yes, been there, done that too with these file based interfaces.

(it's funny how some systems that don't actually process your payment right away actually let you book with a known dummy CC - in Prod. Technically OK because you'll pay for real later - think hotel. Still funny)

Re: Being on call sucks

#257

Post by Aeolun »

hyperman1 wrote: post #211
Aeolun wrote: post #138

> Does anyone have One Weird Trick™ to fix it? My one weird trick is to have a zero tolerance policy for flaky monitors/tests. If it’s not accurate, we either have to drop everything else and fix it, or disable that alarm entirely. Like they say, normalization of deviance is real, and the only way to fight against it is to have every form of deviance be a problem.

I tend to agree with you, but found an exception a while ago. A certain file has to appear before a specific time, or else some people don't get money they deserve and rightfully get very angry. Except 1 or 2 times per year there is nobody in that situation. No payments have to be made. No file appears, as other alerts would signal an empty file. As the relevant time was in business hours and the thing was important,…

No, but, the point is there are never alerts that aren’t alerts.

This is arguably worse, since it’s otherwise a very important alert so anyone that sees it will freak out. Since it happens only 2 times a year, anyone seeing the alert for the first time (depending on churn, this may happen quite often) is going to think it’s really important.

Hopefully you don’t regularly get these alerts because something actually went wrong (say once a year), but that means 66% of all alerts you get are false alarm.

Re: Being on call sucks

#258

Post by tayo42 »

humaniania wrote: post #187

Getting woken up in the middle of the night by a blasting noise is traumatic. I still have PTSD from my on call time. I swear that to subject a prisoner to that type of condition, where they are woken up at arbitrary times and forced to solve complex problems, would be considered cruel and unusual and inhumane treatment.

I got reprimanded and almost fired recently for being pissed at someone who woke me up for some bs page. Like of course ill be angry, i got woken up for work for no real reason. Now im extra pissed about the managers coming down on my for this instead of correcting the oncall...

Re: Being on call sucks

#259

Post by zeckalpha »

sylens wrote: post #34

I have an opportunity to move into an all-remote role that would require me to be on call 24x7 for one week every 3 or 3.5 months. I asked about the frequency of incidents that required the on call person to be pages and it looked like on a bad week, it was about 7 total - a good week was 0. I’m personally torn on whether or not I’ll be okay with the on-call lifestyle so I appreciated this piece for giving me some fo…

Another option would be to be oncall every 1.5 months and get paged between 0 and 3.5 times, by splitting the rotation in two.

Re: Being on call sucks

#260

Post by pm90 »

piva00 wrote: post #25
nickjj wrote: post #19

Earlier quoted context omitted.

Finally, somewhere that I fully agree with. Being on-call while you do not get called upon is 24 / 7 work because you have to live your entire life around being available. Like the blog post mentions, grocery shopping has implications because if you happen to have an on-call event while shopping it means leaving your cart to run to your car to address the situation because you can't leave your house without your work…

> Then there's knowing at any given second your phone can notify you of an event and you have to put the volume at maximum and place it right next to your face every night with an expectation that you could be woken up at any second. And just to pile on that even more: also dreading that you might not wake up if the page/call comes during the heaviest hours of your sleep. It's happened to me, I felt awfully guilty fo…

+1. I am a deep sleeper and Ive missed pages at times that got escalated to the secondary. Nothing major, just something that autoresolved. But the guilt I felt was… incredibly stressful. Because I felt I was letting down people I was familiar with.
Post reply on HN