Money won't answer this problem. Hire someone(s) to cover this is cheaper.
How can I get my engineers to accept being on call?
61–70 of 104 posts
Re: How can I get my engineers to accept being on call?
#62Fuck you, hire an SRE team.
Current employer is trying to have me on call for free 24/7. Not happening.
Re: How can I get my engineers to accept being on call?
#63[dead]
Re: How can I get my engineers to accept being on call?
#64Easy. Pay them a decent salary for every hour they are on call. Simple as that.
As another reply said, you can't afford me.
I did on call 24/7/365 for 23 years. I'll never do it again. My phone goes silent in the evening and doesn't come online again until I get up to come to work. That's never going to change. I'm not 20 anymore, I don't function on no sleep.
Re: How can I get my engineers to accept being on call?
#65Another unspoken red flag is that this app is having so many bugs in production that it requires constant attention from its developers. How are all these bugs getting past QA? Is there a legit development process in place? Who is signing off on such buggy software?
This is what always gets me about these on-call discussions. It's like there is some kind of Stockholm syndrome thing going on where employees hired to write software are forced into working under completely different conditions to what software development should require and no-one ever stops to ask how badly you have screwed up the whole company culture for this issue to be relevant at all.
I recommend investing in real QA and dumping the move-fast-and-break-stuff attitude and probably 90% of the madness that gets bundled under the Agile and DevOps banners as an excuse for not planning or designing anything properly any more. Your Operations team who are running a 24/7 schedule (and hiring for it) anyway should be able to deal with genuine emergencies without requiring non-specialist developers to be on call routinely. The world has worked that way forever and still does outside the DevOps bubble - including numerous industries that operate 24/7 and rely on software to do so.
Re: How can I get my engineers to accept being on call?
#66What's the rationale for setting up the on-call schedule? Is the system that unstable? > on call at least once a week, including the work day and evening, once a month, including the weekend of that week. > I should also note there is a required 15 min response time to pages. The person MUST be able to be online and fixing the problem within 15 mins of the page. I've worked a position where I had extra duties on the…
> Is the system that unstable?
I'm generally anti-on-call, but c'mon, this is an unreasonable standard. No system is 100% perfect, never needing manual intervention to get things going again when something happens.
Any business that has a service that is supposed to be running 24/7 will likely need some form of on-call.
Re: How can I get my engineers to accept being on call?
#67Throwaway so I can talk honestly. I used to work on call in non-software, as a blood analyzer Field Service Engineer for medtech companies, like Siemens, Roche, Beckman-Coulter, Bio-Rad, etc. I spent 5 years doing this. In that industry and job it's standard to work on call rotations that last for 7 days through the weekend and repeat every 3-5 weeks depending on team size. Because the job is hands-on and at customer…
It seems like OP and their CTO would like to have reliable 24/7 operations but do not have the skill to do it by designing systems that won't fail catastrophically out of hours or the budget to do it by hiring dedicated staff to be available at all times to fix problems. So instead they are trying to abuse their existing staff.
The correct response to this IMHO is for the existing staff to collectively tell management where to go. In life you can't always get what you want if you don't have the money or make the effort it requires.
Re: How can I get my engineers to accept being on call?
#68Throwaway so I can talk honestly. I used to work on call in non-software, as a blood analyzer Field Service Engineer for medtech companies, like Siemens, Roche, Beckman-Coulter, Bio-Rad, etc. I spent 5 years doing this. In that industry and job it's standard to work on call rotations that last for 7 days through the weekend and repeat every 3-5 weeks depending on team size. Because the job is hands-on and at customer…
It depends though. I totally see your point and when I was working in a company where we spread the oncall between all people it was a) usually no problem to swap a few hours (we did full weeks, so if you had one thing for 4h one night, usually someone would cover) and b) our response times weren't quite SO harsh - so we did some things, as long as you had cell coverage and a laptop nearby. Of course the cinema was out, or being on the road for hours, but going for groceries was no problem.
I still hated it and don't want to do it anymore because we had so many factors out of our control that made "false" alarms where the only thing to do was check if it was really a problem and then open a ticket upstream at a vendor...
Re: How can I get my engineers to accept being on call?
#69What's the rationale for setting up the on-call schedule? Is the system that unstable? > on call at least once a week, including the work day and evening, once a month, including the weekend of that week. > I should also note there is a required 15 min response time to pages. The person MUST be able to be online and fixing the problem within 15 mins of the page. I've worked a position where I had extra duties on the…
> Is the system that unstable? I'm generally anti-on-call, but c'mon, this is an unreasonable standard. No system is 100% perfect, never needing manual intervention to get things going again when something happens. Any business that has a service that is supposed to be running 24/7 will likely need some form of on-call.
If you have a business that needs to operate 24/7 then you need your Operations team to hire accordingly and then on-call should normally be unnecessary. Operations should be able to fix an emergency that is unsafe, insecure or illegal. Anything else can wait until the next working day even if management might prefer it not to because that costs money.
If a business can't or won't make those arrangements (and pay the bill for them) then maybe its management team need to recognise that fully reliable 24/7 operation is not something they can guarantee and they should plan accordingly.
Re: How can I get my engineers to accept being on call?
#70I've done a fair bit of on-call during my career, but I wouldn't do it anymore (at least not for any dollar amount a company would likely pay). I value my time outside work -- and my sleep -- too much. The standard in SV seems to be that on-call is required and uncompensated (some companies will dishonestly try to suggest that your salary is higher than it would be otherwise). This is gross, and we shouldn't stand fo…
AKA "How can I squeeze employees even harder rather than paying someone to do the work?"