Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Never Ship on Fridays

octomind.dev

Post reply on HN
Print view View original

41–50 of 55 posts

Re: Never Ship on Fridays

#41

Post by PeterStuer »

mvdtnz wrote: post #20

Just another reminder of the absolutely terrible state of our industry. Imagine anyone in any other industry taking this ridiculous stand. "Oh we don't fit roofs on Fridays, in case our roof leaks and we need to bring the roofer back in on the weekend". "We don't do Friday wisdom tooth removals, would hate for the dentist to have to fix a problem in the weekend". "No we don't fit tyres on Fridays, our tyre guy doesn'…

It depends on the type of software. A B2C SaaS that has continuous small releases with easy incremental deployments and rollbacks is incomparable to a bespoke B2B release that integrates with dozens of the customer's other systems.

The latter are typically deployed over the weekend with multiple companies/teams on standby as unexpected interactions are at a certain point of complexity a given.

Your quality aim is for instrumentation and verification to rapidly detect and access and triage issues since assuming you can hit 'deploy' and walk away because your code is perfect (delusional) ignores that 95% of the systems you integrate with aren't even under your control, and 'it worked in test and was signed off' is not an acceptable answer for a client's production system being down on Monday.

Re: Never Ship on Fridays

#42

Post by throw_away_916 »

SideburnsOfDoom wrote: post #3

There are three maturity levels, do not confuse them: 0: We ship to production whenever, why not? If stuff breaks, oops. People make mistakes. 1: We have processes in place to avoid bad releases. These include such measures as people agreeing to avoid Friday releases, and requiring manager approval of releases. 2: The processes are automated, and high enough quality that we can be confident that clicking the "go" but…

At my workplace we release small fixes on Fridays, but prefer to wait until Monday.

I don't see how we could trust ci/cd to catch all errors. Tests are great at detecting expected errors, but it's not possible for us to get 100% confidence.

We also need some sort of tradeoff. Creating a perfect system with few bugs take a lot of time and resources that a startup does not have. We accept that our releases may include bugs, because we can respond to angry customers quickly.

In our B2B industry, most of our customers sales happen during weekends. A bug has a much higher financial impact.

Re: Never Ship on Fridays

#43

Post by SideburnsOfDoom »

cqqxo4zV46cp wrote: post #5
philwelch wrote: post #4

Earlier quoted context omitted.

There’s actually another maturity level: 3. The processes are automated, and high enough quality that we can be confident that clicking the "go" button" will either result in the build/test/deploy pipeline halting due to a problem, or a good release going to production. But we still prefer not to release on Friday, because no CICD methodology is perfect.

There’s another maturity level: blah blah blah, but our CICD setup and overall code quality has long proven itself to be of such quality that we as a team / organisation have opted to assume this risk. My real point is that, at a point, reasonable people can disagree about this, and calling these “maturity levels” strays further from reality and closer to the usual pass-ag nerd habit of mistaking personal preference…

I've seen all three in use, and while agree that "maturity level" is not "absolute objective truth" - that's your overstated reading, but IMHO dismissing them as mere "personal preference" is also wrong in the other direction.

These are not personal - they are are the product of the interactions and agreements of a large number of co-operating people who look for ways to deal with release quality concerns with instruments that they are familiar with: signoffs, manual gates, controls; vs. automation and small batches.

Also dismissing them as "mere preference" implies that the outcomes will be broadly similar, in my experience this is not at all so. The industry has uncovered ways to proceed both fast and safely, but these are not widespread, partly because of the myopia that they are mere "preferences".

Re: Never Ship on Fridays

#44

Post by xhdusux »

> I disagree with Alan on ‘no Friday releases’ being a sign of inadequate quality.

Of course it's a sign of inadequate quality. That's why we don't release on Friday. It's inadequate quality to risk shipping on Fridays.

The problem is that Mr Alan thinks this is a problem when it's actually a conscious trade off we make.

Re: Never Ship on Fridays

#45

Post by SideburnsOfDoom »

ehnto wrote: post #23
master_crab wrote: post #6

Earlier quoted context omitted.

This. Also, you rarely have a full idea how your downstream clients interact with the system that was changed.

Same for me. I think we are all talking like we work on the exact same product. I work at an agency with many codebases, from different clients. Some we inherited that don't yet have good coverage, some with notoriously brittle third party integrations we can't truly test until it is in the live data environment, and some with clients who are very unresponsive and if we planned a Friday deploy the chances of a no-go…

It sounds like you're deep into scenario 2. The mention of "sign off or testers" is a dead giveaway. "testers" being people not automation. likewise "sign-off".

It also sounds like while you know that better exists, the "commercial reality" is "not on this project". I've been there too. I don't see this as being in conflict with what I said above.

Re: Never Ship on Fridays

#46

Post by thiht »

If you can’t release on Friday:

- the quality of your tests sucks, fix that with high priority

- your CI/CD process sucks, fix that quickly

There’s absolutely nothing wrong with releasing on Friday if you’re doing things correctly. And the meme that we shouldn’t ship on Friday needs to die.

Re: Never Ship on Fridays

#47

Post by Ekaros »

As a customer I prefer you not to release on Friday, then go home for weekend and maybe fix stuff on Monday... Specially when you have repeatedly proven that you don't know how to test things you are releasing...

If you call people back to work on Friday after release and keep them at office until things are fixed on your own dime go ahead. Feel free to release on any day.

Re: Never Ship on Fridays

#48

Post by sameerds »

mvdtnz wrote: post #20

Just another reminder of the absolutely terrible state of our industry. Imagine anyone in any other industry taking this ridiculous stand. "Oh we don't fit roofs on Fridays, in case our roof leaks and we need to bring the roofer back in on the weekend". "We don't do Friday wisdom tooth removals, would hate for the dentist to have to fix a problem in the weekend". "No we don't fit tyres on Fridays, our tyre guy doesn'…

Maybe it's just me, but if I knew the dentist will be away tomorrow, there's no way I am going to get my wisdom tooth removed today.

Re: Never Ship on Fridays

#49

Post by SideburnsOfDoom »

winrid wrote: post #39
mvdtnz wrote: post #20

Just another reminder of the absolutely terrible state of our industry. Imagine anyone in any other industry taking this ridiculous stand. "Oh we don't fit roofs on Fridays, in case our roof leaks and we need to bring the roofer back in on the weekend". "We don't do Friday wisdom tooth removals, would hate for the dentist to have to fix a problem in the weekend". "No we don't fit tyres on Fridays, our tyre guy doesn'…

The core of your argument is that it's the worker deciding this, and that they don't want to fix issues over the weekend. That's not true. It's the business deciding that they don't want to ship bugs on Fridays and then take the hit for customers having a bad experience over the weekend if the issue can't be resolved. What if you let someone deploy on a Fri and then they go skiing for the weekend? Sure, someone else…

Indeed, it's the business making the (IMHO poor) decision that the best way to address this issue is more manual oversight, and preventing release at certain times.

There are often other, better ways to address the underlying issue; "no releases on Friday" is a band-aid fix.

Re: Never Ship on Fridays

#50

Post by SideburnsOfDoom »

popularonion wrote: post #15
kenrose wrote: post #14

Earlier quoted context omitted.

Dilbert... or any place that practices continuous delivery? Sounds like your current/previous companies have heavier weight release processes that reduces how often releases go out?

> Sounds like your current/previous companies have heavier weight release processes that reduces how often releases go out? Yes, that probably describes 98% of companies. So what?

> Yes, that probably describes (a very high percentage) of companies. So what?

So good practices are not widely used; therefor the state of the industry on average, is poor.

Post reply on HN