[deleted]
Being on call sucks
111–120 of 282 posts
Re: Being on call sucks
#112I don't mind being on call at my current place. I'm an app owner so I absolutely am the right person to call if things really get out of control. (So I don't even mind being in the escalation list if I'm NOT on call).
What I cannot stand is when folks insist on not tagging errors that they have no intention (or no timeline) on fixing. If you have no plan to fix something, stop waking me up at 3am for an alert. At a former job, I used to put this out to my superiors constantly. "Please tag this alert, get a ticket into our backlog so we can prioritize a fix". Alerts should be for exceptional situations. We've allowed services like New Relic to convince us that Appdex scores always translate to losing money.
Re: Being on call sucks
#113If you're on call 24 hours and you're not getting overtime pay you're being taken advantage of.
Stop giving away free labor. On call is a scam for companies to get away with not staffing properly.
Re: Being on call sucks
#114I've been at my first software engineering gig for 4 years now, we don't have on call, and I've sworn to myself that I will absolutely never take a position with on-call.
I'm a bit worried that it will hamper my career prospects, especially as I've moved to doing more backend work. But I just can't imagine being tied to a work phone on my personal time -- I have a hard enough time enforcing work/life balance as it is.
Re: Being on call sucks
#115So I've been oncall at two major companies (Google and Facebook) and, at least in my experience, this covered both ends of the spectrum. Basically, Google gets it mostly right and Facebook gets it mostly wrong. At Google, a new service has to be supported by the team that developed it. There'a an extensive launch checklist that includes monitoring, having a runbook, etc. Here's the most important part: you're paid wh…
Note that the on-call bonus is not entirely known. I had managers try to put my team "on-call" for a product and after I explained to them how Google actually did it, they suddenly said "oh, it's not really on-call. You just have to be ready to answer the pager at any time and respond".
I was also on what was one of the most dysfunctional on-calls at the company- keeping several distributed clusters of unique business-critical mysql instances that failed frequently with an unreliable failover method. For some reason it was a joint on-call with a neighboring team so I was responsible for systems I didn't know about or understand but were busisness critical. At times it was fun, at times it was educational, but at times, it was the worst thing in the world for me.
Re: Being on call sucks
#116Side note -- I know a lot of early YC startups like to play things fast and loose, but you really should compensate your engineers for after-hours emergencies if they are already working 40 hour weeks. Morally and employee-retention-wise it's the obvious thing to do, but beyond that in certain states and jurisdictions you can easily run afoul of local labor laws if you try to require employees to do things outside of…
Unless you're in a really dysfunctional startup, the employees who voluntarily jump in after-hours to fix OC issues are going to be very quickly compensated with raises and refresher option grants. Far better payout generally than a few bucks of overtime.
It becomes really obvious who is holding the company together and who is coasting, relying on the "senior" (ie, anyone who puts in the effort) engineers to keep the ship afloat, and in a highly competitive engineering market, that will definitely be reflected in equity comp.
Re: Being on call sucks
#117A major issue with on-call, and certainly one I've encountered multiple times, is the high likelihood of moral hazard - the people who are responsible for addressing incidents are not the same people who designed and maintained the system at fault. This results in the former team feeling powerless to put out fires which could have been prevented by more robust design, and the latter team having no incentive to improv…
>> A major issue with on-call, and certainly one I've encountered multiple times, is the high likelihood of moral hazard - the people who are responsible for addressing incidents are not the same people who designed and maintained the system at fault. This results in the former team feeling powerless to put out fires which could have been prevented by more robust design, and the latter team having no incentive to imp…
This is still true today (at AWS). If you write the software, you are ALSO responsible for owning the operations: this means on call.
Re: Being on call sucks
#118I've always been on the ops side so I view being on call as an intrinsic part of my job, regardless of where I work. My current company does it well I think: each team has its own on call rotation covering their own systems (with the ops team I'm on being a backstop) which means that you only get paged for your own bugs. In addition, whoever's on call that week is the designated "person to be disrupted" if someone needs something from your team. If someone needs ops help, they'll go to the ops on call guy first. If I need to ask a question about one of our backend processes, I'll ping the backend on call guy. We have dedicated on call channels where all the alerts and pages go and where people post for help. Most importantly, leadership is just as invested in the product and happy to jump on as we are. The only 4AM page I've had yet, my boss and his boss were on the call before I was. Overall I'm happy with it
Re: Being on call sucks
#119Earlier quoted context omitted.
I was effectively oncall 24/7 at my job at times in 2020. I barely noticed the pandemic. I never strayed far from my computer. Also, yes, I tried not to drink much. I certainly learned what my limits are. People think I am a pretty good engineer (not amazing) but what I am known for is being able to keep that level of performance up for a long time. For my part, despite my reputation, I tried to quit a few times. Not…
Quoted post unavailable.
I think it’s a little disrespectful to resurrect someone’s comment they tried to delete. It’s their comment, and we don’t know their reasons for deleting it.
Re: Being on call sucks
#120I used to be on call too and it sucked. At some point I even bought a personnal pocket laptop specifically so I can still go on bicycle rides when on-call, I also did paddle surfing with phone connected to a bluetooth speaker so I can go back home in case of alert.
Now that I am working for a company who has team members in all timezones this on-call thing doesn't make sense anymore. I understand some companies are not in a global market in term of customers but I don't see why their IT/dev teams couldn't employ people from all over the world. You are much more efficient responding to alerts during your own local office hours than when you were just woken up 30s before in the middle of the night and can barely open your eyes with the laptop backlight on.
Obviously doesn't apply to the one taking care of datacenter duties.