Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Being on call sucks

bobbiechen.com

Post reply on HN
Print view View original

61–70 of 282 posts

Re: Being on call sucks

#61

Post by steveBK123 »

sylens wrote: post #34

I have an opportunity to move into an all-remote role that would require me to be on call 24x7 for one week every 3 or 3.5 months. I asked about the frequency of incidents that required the on call person to be pages and it looked like on a bad week, it was about 7 total - a good week was 0. I’m personally torn on whether or not I’ll be okay with the on-call lifestyle so I appreciated this piece for giving me some fo…

That level of rotation is not a deal breaker in itself. You can basically look at it as 3~4 rotations per year, likely 1 or maybe 2 bad weeks per year.

I've never been on a big enough team to have a rotation that wide, more typical is every 4/6/8 weeks. Beyond that teams usually build out globally and your rotation remains as often but is fewer hours per day (ie - US covers til 8pm when APAC comes in). The bigger concern arguably is how noisy weekends are?

Re: Being on call sucks

#62

Post by zimpenfish »

Broken_Hippo wrote: post #7
cube2222 wrote: post #6

On-call is just fine as long as: 1. There is a rotation. 2. There are few pages, preferably the median should be 0 per week. 3. Spurious / non-actionable alerts get fixed right away (with very high priority) 4. You're not up more than 1 week per 1-1.5 month. 5. You subtract middle of the night pages from your next working day, with bad nights resulting in a day off. Being on-call doesn't mean working overtime. As wit…

6. You are paid an amount for every hour of being on-call. This should be on top of your base pay. This is important. If you are on-call, you do not have the freedom that you would otherwise have and you should be compensated for this.

At one place I worked, they were wanting on-call shifts of 12 hours being no more than 15 minutes away from being logged on (it was a multi-week Big Event) with the promise of, maybe, time off in lieu as recompense.

They were most displeased when I declined this opportunity.

Re: Being on call sucks

#63

Post by NikolaNovak »

dkyc wrote: post #39
oogali wrote: post #26

After years of iteration, here’s what our team does. The team is remote and distributed across multiple time zones ranging from West Coast US to Western Europe. This gets us as close to round the world coverage as we can have. There are two people on call for each shift, each shift lasts a week. It will typically (but not always) be one person from US and one person from UK/EU. This helps reduce the single personal c…

How does it work for you to be on-call 24/7 for escalation? I get that that ends up happening for many committed founders/operators/managers, but I struggle how that can be a real strategy. Are you never off-grid for a bit, or drunk in a bar, or just on a real no-work vacation? There seem to be situations where being on call just isn’t feasible.

(not the OP but my 100 Croatian Lipa)

Depends heavily on how stable app/system is.

I'm from PeopleSoft / ERP world. The same PeopleSoft HCM base application would generate one off-hours pager alert a month at one client; and half a dozen a night at another client; due to different customization/implementation/complexity of business logic and data.

Any on-call/on-shift rotation system must be viewed through the lens of actual demand and need.

At first client, we had 3-4 developers total who shared pagers on weekly basis, as per the OP, with no undue stress or impact on their day job.

At second client, we now have multi-tier support starting with on-shift (junior but specialized ops team members who stare at computer overnight and provide immediate response), Tier 1 and Tier 2 on-call support, and multi-level escalation rotation.

And yes, there are still people who get woken up all the time always, because buck eventually stops there :-/ . Being on call sucks, as per the title. I've been in 24x7 escalation roles; I don't drink to begin with so that's not an issue, but it absolutely had significant negative impact on my social & family life, sleep and stress levels. I've spent significant effort to a) Make the system better, both in terms of more reliable application, and deeper and more self-sufficient support team tree, and b) Move myself out of the role, though that relies on success in a).

I do find it fascinating to occasionally meet very senior people, with family and social life, who are positively EAGER to be on-call and engaged for every little thing all the time always - and then, unfortunately, have same expectations of literally everybody else ("Let's all come on bridge, always, for everything, anytime")

Re: Being on call sucks

#65

Post by g051051 »

hardwaresofton wrote: post #47
g051051 wrote: post #40

> Being On-Call Sucks Which is why I won't do it. Either hire people specifically to do it, or provide sufficient incentives so that people volunteer. Being a good "first responder" is a skill and not everyone has it or wants it.

Yeah I think most of engineering just has never heard of a night crew... And it works out to the benefit of companies, unsurprisingly. If you want people to work around the clock, hire enough people to work around the clock. Maybe this means engineer salaries get repriced, maybe it means more and more companies figure out they don't necessarily need on-call. All that said, the status quo probably won't change, there…

The last place I worked at decided to cut the ops staff entirely and put all devs on support duty. Hilarity ensued because no one wanted to do it and the suits were baffled. They established a hard cutoff date for the transition, and at the last minute had to keep some ops on for a few months. Before that cutoff expired, they laid off almost our entire division, leaving only a fraction of untrained and unwilling staff to keep the lights on.

Even more humor, security hadn't been consulted on this and some requirements when they went to implement it. Any dev, during their 2-week support and on-call period, would have their access to the dev systems removed. Anyone not in their 2-week window would have no access to prod.

Re: Being on call sucks

#66

Post by rajamaka »

Broken_Hippo wrote: post #33
id wrote: post #24

Earlier quoted context omitted.

>It means if you're at your mom's funeral giving a eulogy you leave mid-speech to address PagerDuty. You should not be on-call while you're at your mom's funeral, period.

While I agree totally that you Shouldn't be, we all know that some places will try to force this on you.

The point still stands whether they try to force you or not. I'm constantly amazed by the lack of boundaries some people in this industry entertain.

Re: Being on call sucks

#67

Post by mancerayder »

cube2222 wrote: post #6

On-call is just fine as long as: 1. There is a rotation. 2. There are few pages, preferably the median should be 0 per week. 3. Spurious / non-actionable alerts get fixed right away (with very high priority) 4. You're not up more than 1 week per 1-1.5 month. 5. You subtract middle of the night pages from your next working day, with bad nights resulting in a day off. Being on-call doesn't mean working overtime. As wit…

> 4. You're not up more than 1 week per 1-1.5 month.

That's insane. That's around 20% of your life.

Re: Being on call sucks

#68

Post by ahnberg »

I'm one of the few persons I ever heard of that actually enjoyed being on-call. I believe it goes with my puzzle problem solving mentality to an extent. Being randomly challenged with a problem to look at where you might not know the solution, simply excites me.

Combining on-call duty with an approach of weeding out repeating issues, build better systems and ensuring that unnecessary calls don't happen is key of course, being woken up 25 times for silly predictable errors is pointless and draining.

And finally having an employer that doesn't expect you to be in at 8am if you've been up all night is also very important, catching up on sleep is necessary to manage your balance and health. But given this freedom, I totally dig it. :)

Re: Being on call sucks

#69

Post by [deleted] »

mancerayder wrote: post #67
cube2222 wrote: post #6

On-call is just fine as long as: 1. There is a rotation. 2. There are few pages, preferably the median should be 0 per week. 3. Spurious / non-actionable alerts get fixed right away (with very high priority) 4. You're not up more than 1 week per 1-1.5 month. 5. You subtract middle of the night pages from your next working day, with bad nights resulting in a day off. Being on-call doesn't mean working overtime. As wit…

> 4. You're not up more than 1 week per 1-1.5 month. That's insane. That's around 20% of your life.

[deleted]

Re: Being on call sucks

#70

Post by kayodelycaon »

On-call is even worse for people with disabilities. I quite literally can't do it unless I stop taking my antipsychotic.

Under ADA, I can not be placed on call, regardless of policy, nor can I be discriminated against for that. On-call is not an essential function of being a software developer, with very few exceptions—all of which have nothing to do with "policy" or "fairness".

Needless to say, companies (and some coworkers) really don't like this.

Post reply on HN