Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Being on call sucks

bobbiechen.com

Post reply on HN
Print view View original

171–180 of 282 posts

Re: Being on call sucks

#171

Post by [deleted] »

4ggr0 wrote: post #133
jddddd wrote: post #123

Earlier quoted context omitted.

It won't. On call is a symptom of poorly run company. It's a great signal that you should run far away from any place that requires it. Most software isn't as critical as we think, and the software that is, is expensive enough to have a properly sized staff. On call exists for the same reason game devs are paid shit and open source exists... Software engineers don't value themselves properly and love giving away free…

This sounds like a very software-engineery opinion. Not saying that you're wrong, I understand that on-call is not that important in software development. But as a System Administrator I think that on-call is useful, else we wouldn't notice any outages happening during the night and would then only start working on it in the morning. Plus, we have customers who actually work during the night, be it timezones or speci…

[deleted]

Re: Being on call sucks

#172

Post by muttled »

di4na wrote: post #55

The one weird trick is to pay people to fix it. There is an incentive problem here. The second one weird trick is to legitimately ask people about the on call experience. Again. And again. And again. The reason nothing change is because noone is incentivised to change it.

Or it's incentivized in the wrong direction, in the case of something like an MSP that bills more for out-of-hours calls so it's actually profitable to have events happen.

Re: Being on call sucks

#173

Post by morelisp »

kazen44 wrote: post #80
morelisp wrote: post #22

Earlier quoted context omitted.

This is incredible; at our German company we get 0.1x/1.5x. (Though we also don't promise response times that would interrupt e.g. grocery shopping.)

having resonable response times make a huge difference in being oncall. in my experience, most employer seem to think responding within 30minutes or so is doable, and i tend to agree with them. Usually makes it possible for you to just do things around the house/ in your life, and not having to rush back home when you run into an oncall issue.

In our case it's one hour, essentially based on "what if my home internet doesn't work and I have to get to the actual office to start responding, how long will it take me to get there from anywhere in the city." You don't go on vacation or plan your wedding, but otherwise your day is pretty normal.

I can't imagine less than 30 minutes.

Re: Being on call sucks

#174

Post by muttled »

Fire-Dragon-DoL wrote: post #37
gc22browsing wrote: post #23

Earlier quoted context omitted.

I used to work for a small systems integrator. We used on-call mobile phones, the "hot potato" was carried by the on-call engineer. Any actual time worked outside of core-working hours was paid back in the form of time-off-in-lieu. The salary generously reflected the being on-call requirement. The other factor was that a "call-out" was not completed until the root cause was fixed. I believe the real reason for the co…

I don't think time off is enough,unless with a multiplier. 1 hour of on call task at 4am will affect the entire day, possibly the next one

And your health. Getting woken up often is not good for your heart!

Re: Being on call sucks

#175

Post by coffeefirst »

Aeolun wrote: post #138

> Does anyone have One Weird Trick™ to fix it? My one weird trick is to have a zero tolerance policy for flaky monitors/tests. If it’s not accurate, we either have to drop everything else and fix it, or disable that alarm entirely. Like they say, normalization of deviance is real, and the only way to fight against it is to have every form of deviance be a problem.

Yeah. Also, unless you are a genuinely essential application—like air traffic control, a hospital, or a nuclear power plant—you can live with a few hours of downtime.

AWS goes on the fritz for a few days out of of every year and breaks half the internet. Your business will be okay.

Re: Being on call sucks

#176

Post by angarg12 »

bobkazamakis wrote: post #137
angarg12 wrote: post #129

This article should be called "being on a sucky team sucks". It sounds like OP just has experience with one bad on call. I've also been there. I even heard of teams with +100 high severity issues per week. But it doesn't need to be. My current team has the best on call I've ever experienced. It's a mix of a lucky product and some discipline. It isn't rocket science really. If a team is drowning in ops here are two ea…

>1. Whoever is on call, their job isn't only to answer pages, but also to improve the system Way to make being on-call more punishing as well as detract from both goals -- your clients receive worse support and the amount of work done on the fix is inversely proportional to the time you're spending with clients who are calling you because there is a problem, not because they just want to talk. 2 actually works; tech…

The point of 1 is that whoever is oncall shouldn't be working on features/sprint tasks, but actually fixing the underlying problems of the system. This is second to addressing the pages.

Of course if your system is so deficient that engineers spend 100% of their time putting out fires, then you need to address that first. That's what reducing toil from the SRE books is about.

Re: Being on call sucks

#177

Post by nickd2001 »

One way to make on-call less bad, ask your employer to let you work normal 8hr days on the weekend that you're on call, in return for having 2 weekdays off the next week. That way you don't have any weekend days that are lost due to having to be available. I was allowed to do that, it was a win-win. Useful for them to have really thorough extra cover over weekend, then 2 uninterrupted days in the hills for me. ;)

Re: Being on call sucks

#178

Post by morelisp »

zeckalpha wrote: post #72

> Last year as part of my move to product management, I was removed from the pager rotation for my old team. Product should be on oncall rotation, too, even if it is as a shadow. This is an important feedback mechanism about the choices they make.

We are even in the process of spinning up some product-only oncall for issues that don't need dev "escalation" (e.g. most commonly some customer messed up their credentials or their API token expired end-of-day, resulting in some higher error rate).

Re: Being on call sucks

#179

Post by mlyle »

HelloNurse wrote: post #164
sokoloff wrote: post #125

Earlier quoted context omitted.

What I find interesting is that if you give employees a choice of 100 units of currency per year as salary plus 1 unit of currency for each week on call with an expectation of 1 week on call every four weeks versus 115 units of currency for the year with the same on-call expectations, some employees will feel better about the first arrangement and some will feel better about the second arrangement. Some people want t…

What do you mean by "expectation"? One day on call, one day paid overtime; anything else is abusive. There should be no incentive to require or avoid "free" work, only advance planning of who'll be on call (with more available engineers doing more turns).

> What do you mean by "expectation"? One day on call, one day paid overtime; anything else is abusive.

I don't think being available for work if an unlikely emergency comes up usually justifies the same comp as actively working that entire time.

Re: Being on call sucks

#180

Post by ectopod »

sam0x17 wrote: post #100

Side note -- I know a lot of early YC startups like to play things fast and loose, but you really should compensate your engineers for after-hours emergencies if they are already working 40 hour weeks. Morally and employee-retention-wise it's the obvious thing to do, but beyond that in certain states and jurisdictions you can easily run afoul of local labor laws if you try to require employees to do things outside of…

That's only half the story. You should also compensate employees for being on call. As the article notes, it is a significant chore.
Post reply on HN