Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Breaking Up with On-Call

reflector.dev

Post reply on HN
Print view View original

11–20 of 202 posts

Re: Breaking Up with On-Call

#11

Post by strken »

danpalmer wrote: post #5

As a current SRE, and having worked in a small startup, this doesn't echo my experience at all. What the author describes is possible what we would call "on duty" work, the grunt/maintenance work that comes with big software systems. It's not fun, and most companies/teams have friction getting this sort of work done. It's also however not how my SRE role is defined by any stretch. Our on-call work is much more about…

Yeah, I was similarly confused. My experience has been that on-call is a roster of who will keep their laptop with them on the weekend. Incidents involve little to no customer comms, because if anything more than a sentence once an hour is necessary then there's someone else on call to handle it.

Support work is a necessary nuisance, but it's also not what on-call is meant to be.

Re: Breaking Up with On-Call

#12

Post by tacticus »

pokoleo wrote: post #3

From my experience working on SaaS, and improving ops at large organizations, I've seen that "on-call culture" often exists inversely proportional to incentive alignment. When engineers bear the full financial consequences of 3AM pages, they're more likely to make systems more resilient by adding graceful failure modes. When incident response becomes an organizational checkbox divorced from financial outcomes and pla…

IMO it's when the incident response and readiness practice imposes a direct backpressure on feature delivery that you get the issues actually fixed and a resilient system.

if it's just the engineer while product and management see no real cost then people burn out and leave.

> The most successful teams I've seen treat on-call like a leading indicator - every incident represents unpriced technical debt that should be systematically eliminated. Each alert becomes an investment opportunity rather than a burden to be rotated.

100%

Re: Breaking Up with On-Call

#13

Post by abrookewood »

caseyohara wrote: post #2

Not to distract from the article, but I'm pretty sure that photo of the guard tower is from Manzanar, one of the concentration camps in California that interned Japanese Americans during WWII. Probably not in great taste to use that photo to represent the idea of "guard duty" in software. https://en.wikipedia.org/wiki/Manzanar

[flagged]

Re: Breaking Up with On-Call

#14

Post by rednafi »

A moderate on-call ritual is a necessary evil. I’ve worked at places that tried to get rid of it with all kinds of automation and playbooks, only to revert back to PagerDuty a few months later.

That said, my last workplace completely burned me out with a terrible on-call policy and an absurdly short recovery period. Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of normal work and didn’t warrant additional compensation.

I get why it’s necessary, but I’m lucky enough to be in a position where I can flat-out decline any job that mandates unpaid on-call.

Re: Breaking Up with On-Call

#15

Post by rednafi »

pokoleo wrote: post #3

From my experience working on SaaS, and improving ops at large organizations, I've seen that "on-call culture" often exists inversely proportional to incentive alignment. When engineers bear the full financial consequences of 3AM pages, they're more likely to make systems more resilient by adding graceful failure modes. When incident response becomes an organizational checkbox divorced from financial outcomes and pla…

> When engineers bear the full financial consequences of 3AM pages, they're more likely to make systems more resilient by adding graceful failure modes.

Making engineers handle 3 AM issues caused by their code is one thing, but making them bear the financial consequences is another. That’s how you create a blame-game culture where everyone is afraid to deploy at the end of the day or touch anything they don’t fully understand.

Re: Breaking Up with On-Call

#16

Post by lolinder »

rednafi wrote: post #14

A moderate on-call ritual is a necessary evil. I’ve worked at places that tried to get rid of it with all kinds of automation and playbooks, only to revert back to PagerDuty a few months later. That said, my last workplace completely burned me out with a terrible on-call policy and an absurdly short recovery period. Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of nor…

> upper management tried to gaslight everyone into thinking on-call was just part of normal work and didn’t warrant additional compensation.

If it's made explicitly clear in the job interview process that on call is expected in a role and if it's enforced evenly across the organization, this can absolutely be normal and ethical—you're on a specific salary and your salary includes on call expectations. If you knew about that going in then you weighed it against the offered salary and decided it was worth it. It's not "unpaid on-call" in that case, it's paid on-call.

Obviously it can also be done poorly in ways that are harmful and dishonest. Not communicating it clearly during the job interview and salary negotiation, applying it inconsistently, or changing the frequency or the difficulty of the rotation after you have started are all real problems.

Re: Breaking Up with On-Call

#17

Post by rednafi »

lolinder wrote: post #16
rednafi wrote: post #14

A moderate on-call ritual is a necessary evil. I’ve worked at places that tried to get rid of it with all kinds of automation and playbooks, only to revert back to PagerDuty a few months later. That said, my last workplace completely burned me out with a terrible on-call policy and an absurdly short recovery period. Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of nor…

> upper management tried to gaslight everyone into thinking on-call was just part of normal work and didn’t warrant additional compensation. If it's made explicitly clear in the job interview process that on call is expected in a role and if it's enforced evenly across the organization, this can absolutely be normal and ethical—you're on a specific salary and your salary includes on call expectations. If you knew abo…

I agree with your expansion. In this particular case, the on-call policy was enforced after the hiring.

Re: Breaking Up with On-Call

#18

Post by AdieuToLogic »

lolinder wrote: post #16
rednafi wrote: post #14

A moderate on-call ritual is a necessary evil. I’ve worked at places that tried to get rid of it with all kinds of automation and playbooks, only to revert back to PagerDuty a few months later. That said, my last workplace completely burned me out with a terrible on-call policy and an absurdly short recovery period. Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of nor…

> upper management tried to gaslight everyone into thinking on-call was just part of normal work and didn’t warrant additional compensation. If it's made explicitly clear in the job interview process that on call is expected in a role and if it's enforced evenly across the organization, this can absolutely be normal and ethical—you're on a specific salary and your salary includes on call expectations. If you knew abo…

>> upper management tried to gaslight everyone into thinking on-call was just part of normal work and didn’t warrant additional compensation.

> If it's made explicitly clear in the job interview process that on call is expected in a role and if it's enforced evenly across the organization, this can absolutely be normal and ethical—you're on a specific salary and your salary includes on call expectations.

Where this logic breaks down is when on-call expectations are in addition to what a salaried position compensates - a 40-ish hour work week.

Expecting an employee to be paid industry rate for services rendered and then expecting periodic 24-168 hour near real-time availability without compensation is, by definition, "unpaid on-call".

Re: Breaking Up with On-Call

#19

Post by bigstrat2003 »

rednafi wrote: post #14

A moderate on-call ritual is a necessary evil. I’ve worked at places that tried to get rid of it with all kinds of automation and playbooks, only to revert back to PagerDuty a few months later. That said, my last workplace completely burned me out with a terrible on-call policy and an absurdly short recovery period. Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of nor…

> Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of normal work and didn’t warrant additional compensation.

Every position I've held (save my current one), that is most definitely the norm. If you have a busy night managers are cool with you coming in late the next day (or potentially not at all), but it's very unusual to be paid for on call in my experience.

Re: Breaking Up with On-Call

#20

Post by tbrownaw »

rednafi wrote: post #14

A moderate on-call ritual is a necessary evil. I’ve worked at places that tried to get rid of it with all kinds of automation and playbooks, only to revert back to PagerDuty a few months later. That said, my last workplace completely burned me out with a terrible on-call policy and an absurdly short recovery period. Not to mention, upper management tried to gaslight everyone into thinking on-call was just part of nor…

> tried to gaslight everyone into thinking on-call was just part of normal work and didn’t warrant additional compensation.

It is part of normal work for roles where it's a thing, and I'd think it could be either baked in or on top as long as which one it was was known at salary negotiation time.

Post reply on HN