Breaking Up with On-Call • Hacker News Board

Re: Breaking Up with On-Call

#51

Post by anal_reactor » Tue, Mar 18, 2025, 8:20 AM UTC

cdnthrownawy39 wrote: post #48

coolgoose wrote: post #28

Earlier quoted context omitted.

Sounds like normal support work not sure why that's affecting morale. It's normal in any kind of system to also cover issues. Yes, at some point you might want to have a customer support / customer success later to at least triage them, but that makes more sense as you get bigger not when you are small. I actually like having discussions on support days with customer. Yes, sometimes they're more annoying but it's dir…

My on call experience required that I had to be able to respond within 10 ten minutes of the call, with 24/7/365 coverage. But if I couldn't get the issue resolved remotely it meant that I'd have to be in the office lab to recreate and reproduce the problem. It effectively restricted my movements personal movements to stay within commute distance of my office, and that includes all my vacation time as well. That was…

I don't understand why on-call is normal. It's a huge mental burden on employees, which is especially an important issue in modern times of widespread mental issues, just so that some shitty mobile app can be available 24/7/365. If your business is important enough to have on-call, then you should have dedicated employees covering night shifts and nothing else, effectively limiting it to someone's office hours, effectively removing on-call. I think that there should be laws against on-call.

Re: Breaking Up with On-Call

#52

Post by bayindirh » Tue, Mar 18, 2025, 8:25 AM UTC

anal_reactor wrote: post #51

cdnthrownawy39 wrote: post #48

Earlier quoted context omitted.

My on call experience required that I had to be able to respond within 10 ten minutes of the call, with 24/7/365 coverage. But if I couldn't get the issue resolved remotely it meant that I'd have to be in the office lab to recreate and reproduce the problem. It effectively restricted my movements personal movements to stay within commute distance of my office, and that includes all my vacation time as well. That was…

I don't understand why on-call is normal. It's a huge mental burden on employees, which is especially an important issue in modern times of widespread mental issues, just so that some shitty mobile app can be available 24/7/365. If your business is important enough to have on-call, then you should have dedicated employees covering night shifts and nothing else, effectively limiting it to someone's office hours, effec…

It's similar to using Electron to develop a software: Create quickly, offload the inefficiencies to the users' computers, so the developer can be comfy, and the development can be cheap.

When you have on-call like this, you offload your expenses to your employee's life and mental health, and it's cheaper on the paper, for the short term.

Then your company sinks as people starts leave, and the managers ask "Why?".

Re: Breaking Up with On-Call

#53

Post by baq » Tue, Mar 18, 2025, 8:27 AM UTC

cdnthrownawy39 wrote: post #48

coolgoose wrote: post #28

Earlier quoted context omitted.

Sounds like normal support work not sure why that's affecting morale. It's normal in any kind of system to also cover issues. Yes, at some point you might want to have a customer support / customer success later to at least triage them, but that makes more sense as you get bigger not when you are small. I actually like having discussions on support days with customer. Yes, sometimes they're more annoying but it's dir…

My on call experience required that I had to be able to respond within 10 ten minutes of the call, with 24/7/365 coverage. But if I couldn't get the issue resolved remotely it meant that I'd have to be in the office lab to recreate and reproduce the problem. It effectively restricted my movements personal movements to stay within commute distance of my office, and that includes all my vacation time as well. That was…

You should be paid extra for all that time and indeed there are countries with legal frameworks which require an employer to pay the employee if their personal life outside of working hours is in any way constrained. If you are a contractor, always put limits for this kind of extra services.

Re: Breaking Up with On-Call

#54

Post by rohgpt » Tue, Mar 18, 2025, 8:44 AM UTC

I used to be on-call too. I still have muscle memory of the buzz that’d make me leap off my couch like my house was on fire. Except it wasn’t. Just prod, doing prod things.

We all know it. You trade your peace of mind, sleep, and sometimes personal freedom (hello, “don’t go beyond cell signal” life). And I agree with many of you—it shouldn’t be this way.

We’ve seen teams turn their on-call rotations into something way more humane. Not perfect (nothing in ops is, let’s be real), but sustainable.

At Zenduty, we work with a lot of SREs, DevOps folks, and incident commanders who’ve said, “We can’t keep doing this.” So they started rethinking on-call. One of our partners cut their alert volume by 50%. Half! That’s not a magic trick. Just better processes, better tooling, and some good old-fashioned empathy for the humans behind the screens.

And if your team’s not giving you breathing room after an all-nighter? You deserve better. I’ll just say it.

I saw some of you mention how this stuff becomes political or gets deprioritized in favor of shiny new features. Ugh. I feel that in my bones. The best teams we’ve worked with treat incidents as opportunities to make things better. Every alert is basically your system saying “Help me, I’m tired.” And when you listen and fix it? Less 3 AM wake-ups. Win-win.

Anyway, I didn’t mean to ramble, but this topic is close to home. If you’re still stuck in on-call hell, or just want to vent about it. Always happy to swap war stories or chat about how folks are making on-call suck less.

Stay sane out there, Rohan DevRel @ Zenduty

Re: Breaking Up with On-Call

#55

Post by chgs » Tue, Mar 18, 2025, 8:50 AM UTC

rednafi wrote: post #14

A moderate on-call ritual is a necessary evil. I’ve worked at places that tried to get rid of it with all kinds of automation and playbooks, only to revert back to PagerDuty a few months later. That said, my last workplace completely burned me out with a terrible on-call policy and an absurdly short recovery period. Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of nor…

My company has operated a 24/7 shift rota for 100 years. American “On call” is not normal.

Some companies might have exceptional on call - the C suite will get called back from holidays or whatever should things really hit the fan. That’s once every few years.

Re: Breaking Up with On-Call

#56

Post by whstl » Tue, Mar 18, 2025, 8:51 AM UTC

cdnthrownawy39 wrote: post #48

coolgoose wrote: post #28

Earlier quoted context omitted.

Sounds like normal support work not sure why that's affecting morale. It's normal in any kind of system to also cover issues. Yes, at some point you might want to have a customer support / customer success later to at least triage them, but that makes more sense as you get bigger not when you are small. I actually like having discussions on support days with customer. Yes, sometimes they're more annoying but it's dir…

My on call experience required that I had to be able to respond within 10 ten minutes of the call, with 24/7/365 coverage. But if I couldn't get the issue resolved remotely it meant that I'd have to be in the office lab to recreate and reproduce the problem. It effectively restricted my movements personal movements to stay within commute distance of my office, and that includes all my vacation time as well. That was…

Yeah, I worked in a couple companies that had the same requirements.

It also meant having to wake up on command in case the phone beeped, being unable to drink in my free time, being limited in which activities I could go to.

It was hell.

I know people are gonna hit back with "you're doing it wrong", but in this case it's the company doing it wrong, but nobody on HN will go there and tell them.

Re: Breaking Up with On-Call

#57

Post by whstl » Tue, Mar 18, 2025, 9:07 AM UTC

AdieuToLogic wrote: post #26

The industry myth of "devs need to be on-call just in case prod crashes at 3am" needs to die. First, a system failure addressed by someone awoken "at 3am" assumes said person can transition from sleep to peek analytic mode in moments. This is obviously not the case. Second, a system failure addressed by someone awoken "at 3am" assumes said person possesses omniscient awareness of all aspects of a non-trivial system.…

Great points.

> If a resolution requires a system change, what organization would allow one person at 3am to make it and push to prod?

The only advantage of having a micromanaging manager: if they want to review all PRs and system changes, they also must be on call every single night of the year.

Re: Breaking Up with On-Call

#58

Post by genocidicbunny » Tue, Mar 18, 2025, 9:14 AM UTC

lolinder wrote: post #16

rednafi wrote: post #14

A moderate on-call ritual is a necessary evil. I’ve worked at places that tried to get rid of it with all kinds of automation and playbooks, only to revert back to PagerDuty a few months later. That said, my last workplace completely burned me out with a terrible on-call policy and an absurdly short recovery period. Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of nor…

> upper management tried to gaslight everyone into thinking on-call was just part of normal work and didn’t warrant additional compensation. If it's made explicitly clear in the job interview process that on call is expected in a role and if it's enforced evenly across the organization, this can absolutely be normal and ethical—you're on a specific salary and your salary includes on call expectations. If you knew abo…

For me personally, there's no possible way that a company can correctly factor on-call costs into my salary, because the on-call costs are based on what I am doing at the time.

You'd have to pay me a biblically large multiplier on my base salary to get me to pick work over my kids or wife. My kids aren't going to remember that I made a few thousand bucks by ducking out halfway through an important-to-them event. My wife is sure as shit ain't going to forgive me _or you_ if you call me into work in the middle of some adult time. And the cost for you to make sure that I'm not participating in such activities and am available to work on very short notice is...well, it's the exact same number as if you had interrupted such activities.

If you really need the coverage in the middle of the night, hire more engineers whose normal 8-ish hour shifts are during the night? Can't afford to? Perhaps you're not so good at business then. Better yet, stop building a fucking house of cards that can topple your company if it wobbles a little bit. If your systems going down for a few hours can take down your company, outside of some exceptional circumstances, that's a failure on your part.

Re: Breaking Up with On-Call

#59

Post by whstl » Tue, Mar 18, 2025, 9:14 AM UTC

nijave wrote: post #31

DanHulton wrote: post #22

Earlier quoted context omitted.

This assumes that the engineers in question get to choose how to allot their time, and are _allowed_ to spend time to add graceful failure modes. I cannot tell you how many stories I have heard of, and companies I have directly worked at, where this power is not granted to engineers, and they are instead directed to "stop working on technical debt, we'll make time to come back to that later". Of course, time is never…

Definitely an issue but I think there's a little room for push back. Work done outside normal working hours is automatically the highest priority, by definition. It's helpful to remind people of that. If it's important enough to deserve a page, it's top priority work. The reverse is also true (if a page isn't top priority, disable the paging alert and stick it on a dashboard or periodic checklist)

You're right, but it's still outrageous that engineers need to burn political capital in order to have proper sleep and avoid burnout.

Re: Breaking Up with On-Call

#60

Post by modo_mario » Tue, Mar 18, 2025, 10:21 AM UTC

Xylakant wrote: post #40

brigandish wrote: post #23

Earlier quoted context omitted.

Who, specifically, would that picture offend or change protect?

People that have been interned in this or one of the other camps, or their descendants. It’s one generation ago, people born and raised in camps are still alive. George Takeo for example was born in one of the segregation camps. It’s a low stakes change. Nobody assumes harmful intentions from the author - I would not have recognized the picture either. But now that it’s been pointed out that it’s from a site where pe…

>Nobody assumes harmful intentions from the author

Then what's the issue or purpose? Other than satisfying or inflating some own moral selfimage? That kind of social dynamic appears to be what this kind of things is about most of the time as opposed to preventing mental harm or stopping such issues from (re)occurring.

>Knowing this, and insisting on keeping the images is now willfully associating with harmful behavior.

Do you not associate with harmfull behavior in far far more direct ways? Perhaps something that's considered ridiculous for most people to avoid like paying taxes that get spent on bombs or the like?

If i want to make an article showing a prison and i pick one from google does it matter if that prison happened to be used in the past to house civil rights activists? Is the implication that that somehow normalises, advocates for or otherwise inches towards oppressing civil rights movements in any meaningfull way?