Poll: How often do you deploy to production? • Hacker News Board

Re: Poll: How often do you deploy to production?

#61

Post by sangupta » Thu, Jun 27, 2013, 12:57 PM UTC

At http://multiplx.com (an RSS Reader) we usually do a production release once every day or two. This helps us keeping the bugs minimal and users happy.

Though it consumes a lot of bandwidth, but automation is the key here. It can be minimized if you have a CI tool that can build the product and deploy on the machine itself.

Re: Poll: How often do you deploy to production?

#62

Post by thraxil » Thu, Jun 27, 2013, 1:07 PM UTC

We've got a CI pipeline that watches the git repo, runs the tests, does an automatic deploy to a staging server, runs smoke tests against that, and merges that code into the "prod" branch. Then there's a simple web interface with a big button that checks out prod and deploys it to production. So it's almost continuous deployment; just one point of manual review involved between the 'git push' and code running on production.

Re: Poll: How often do you deploy to production?

#63

Post by bradleyland » Thu, Jun 27, 2013, 1:39 PM UTC

This is an interesting enough question if you're just curious, but I wouldn't base any decisions on it. It's a good example of asking the wrong question. Different products require different approaches to QA and deployment.

For example, our software is sold to enterprise customers. These customers use the software to run real-time reverse auction procurements where the total value of the purchase can be anywhere from $250,000 to $20,000,000.

The results of the reverse auction events are awarded in the form of a contract between buyer and seller. If a software bug causes an incorrect calculation, it's a big problem (0.1% error on a $10M purchase is a $10,000 error). And by big problem, I mean that weeks (maybe months) worth of effort from our team, the customer's team, and several vendors go down the drain.

Put simply, a bug in our bid core could cost us $40,000-$50,000, assuming the lost business assessment is limited to the single failed procurement event. Looking at the total value of a lost customer, you're easily talking $250k.

Because of this, we have a very long QA cycle. Outside of automated tests, our software is touched by humans (a lot) before it goes to production, and production releases are less frequent (once every couple of months).

The frequency with which you deploy, and the amount of effort you put in to QA is determined by the cost of failure. We're in a rather unenviable position of being a small fish in the enterprise market, so our cost of failure is huge (we have a small number of huge clients). Thus, we invest significant effort in QA and slow-roll to production. Your situation may be entirely different.

Re: Poll: How often do you deploy to production?

#64

Post by andyhmltn » Thu, Jun 27, 2013, 1:47 PM UTC

pcowans wrote: post #48

andyhmltn wrote: post #46

Earlier quoted context omitted.

It isn't causing any problems! But in a recent thread on git deployment, there was a tonne of hate for this method

... one other thing though. With a bit of practice you can remove the need to develop on feature branches; break things down into really small chunks and get into the habit of making half-done features unobtrusive (e.g. hiding UI elements until they're ready). That way you can push to master often and have confidence it'll always be safe to deploy, which simplifies your tooling and reduces the scope for merge conflic…

I personally find branching a lot better than the method you described. Master is always safe to deploy in my current setup and I don't usually get merge conflicts :-)