Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Poll: How often do you deploy to production?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 64 posts

Re: Poll: How often do you deploy to production?

#51

Post by StavrosK »

buro9 wrote: post #50
StavrosK wrote: post #43

Earlier quoted context omitted.

Deploying frequently when you're 1-2 developers is much easier, usually. A very important predictor of code quality that I've found is gut feeling, and, when I feel pretty confident about a piece of code, it usually turns out fine. There have been times when I said "hmm, this 3-line change doesn't feel right" and it turned out to have a bug in it, so I rely on intuition a lot (tests can't catch everything). When you'…

Oh, the government issue was that they would outsource management of the data centers and server management. The incentives in these outsourced contracts rewarded uptime and little else, and the best way to ensure that was to penalise anything that threatened it... such as a new release. The end result, is that to have a patch deployed you'd have to get the patch, the whole codebase, and everything replicated and pro…

I'll file this under "corporate horror stories" :)

I was more referring to how easy it is to push frequently to production when you have a small team of good developers, though. As the team gets larger, the ease of pushing to production drops (leading to your example above, in the long run and with crappy dev culture, I guess).

Re: Poll: How often do you deploy to production?

#52

Post by bobsy »

aidos wrote: post #25
bobsy wrote: post #5

With my employer we deploy several times a day. We have 2 SVN repos. One dev, one live. To make a change live you commit your change to the live repo and it is auto-synced. I am not sure how we came to have this. When I first joined there was a code reviewer who manually pushed changes live. He quit and the server admin made it auto. I wouldn't recommend this set up. I am incredibly competent. I make maybe 1 minor mi…

Out of interest, how big is the team?

3 developers, 1 server admin, 5 other who do marketting, design, etc

Re: Poll: How often do you deploy to production?

#53

Post by brokentone »

The majority of my SVN/deploy magic happens with hooks. Pre-commit running PHP Lint - Keeps out the stupid errors that we all make. Post-commit svnlook (diff) emailed to the full dev group - dead simple code review Post-commit svn up of dev site Post-commit varnish clear of dev site Post-commit commit message in hipchat Deploy message in hipchat

Re: Poll: How often do you deploy to production?

#54

Post by anton_gogolev »

bobsy wrote: post #5

With my employer we deploy several times a day. We have 2 SVN repos. One dev, one live. To make a change live you commit your change to the live repo and it is auto-synced. I am not sure how we came to have this. When I first joined there was a code reviewer who manually pushed changes live. He quit and the server admin made it auto. I wouldn't recommend this set up. I am incredibly competent. I make maybe 1 minor mi…

Two questions.

1. Why bother with two SVN repos? 2. Any specific reason for SVN at all (large files, binary files, non-technical users)

Re: Poll: How often do you deploy to production?

#55

Post by bobsy »

wrath wrote: post #38
bobsy wrote: post #5

With my employer we deploy several times a day. We have 2 SVN repos. One dev, one live. To make a change live you commit your change to the live repo and it is auto-synced. I am not sure how we came to have this. When I first joined there was a code reviewer who manually pushed changes live. He quit and the server admin made it auto. I wouldn't recommend this set up. I am incredibly competent. I make maybe 1 minor mi…

At my company I tend to think that we hire very competent developers. Our code base is so complicated (because of legacy code and/or the complexity of the tasks) that developers make mistakes all the time. They are caught by both a peer code-review process and the QA team or support/customers. I trust that you are "incredibly competent" but 1 mistake a month seems very low to me. But what do I know! :) What type of p…

> I trust that you are "incredibly competent" but 1 mistake a month seems very low to me. But what do I know! :)

Honestly, I don't know where "incredibly" came from. I was going for pretty (pretty competent). I think my mind was in 2 places at once.

I have been working with the same code base for 2 years. I know it back to front. I am good at testing and my mind is open to the various scenarios which need to be tested for each change which is made. I make mistakes all the time but they very very very rarely sneak out and get live.

> What type of product do you have?

Its a website-as-a-service product.

> Do you have a QA team?

Nope. For major features we try to get all employee's to pitch in for a group test on a development server. That is the best we do. Minor features / fixes are just tested locally by the developer then go out.

> Do you track new and close bug counts from week to week?

We use bug tracking software. We track the amount of open bugs. When I started I was getting around 20 bugs / developer support requests a week. Now I get one or two. Most of what I get through the tracker is trouble shooting unfamiliar hosting configurations.

> Similarly do you track regression counts from build to build?

No

> Does the code base have a high completeness rate for unit tests?

The code base doesn't have unit tests. It was built by a hobbiest 4 years ago. The current version supports legacy features from a previous version. It is mainly procedural with a smattering of random classes. There is a lot of code duplication. It is the definition of spaghetti code.

Basically, the core software needs to be rewritten. This isn't a management priority so we make do.

> Do you only deploy if you pass the unit tests?

N/A

> What if you introduce a bug that screws up the data, do you have a fallback strategy?

We have twice daily backups and can rollback.

Re: Poll: How often do you deploy to production?

#56

Post by bobsy »

anton_gogolev wrote: post #54
bobsy wrote: post #5

With my employer we deploy several times a day. We have 2 SVN repos. One dev, one live. To make a change live you commit your change to the live repo and it is auto-synced. I am not sure how we came to have this. When I first joined there was a code reviewer who manually pushed changes live. He quit and the server admin made it auto. I wouldn't recommend this set up. I am incredibly competent. I make maybe 1 minor mi…

Two questions. 1. Why bother with two SVN repos? 2. Any specific reason for SVN at all (large files, binary files, non-technical users)

1. I wasn't around when the system was set up. I think the idea was that developers could work together on an incremental version in dev repo. Then it could be pushed out as a clear update in the live repo.

2. It was what someone chose 4 years ago. The guy has since left. Everyone is familiar with SVN. SVN is pretty easy to get into. We haven't seen a need to change.

Re: Poll: How often do you deploy to production?

#57

Post by bobsy »

pcowans wrote: post #33
bobsy wrote: post #5

With my employer we deploy several times a day. We have 2 SVN repos. One dev, one live. To make a change live you commit your change to the live repo and it is auto-synced. I am not sure how we came to have this. When I first joined there was a code reviewer who manually pushed changes live. He quit and the server admin made it auto. I wouldn't recommend this set up. I am incredibly competent. I make maybe 1 minor mi…

So, obvious question... why do you not have tests in this set up?

I answered this above.

The code base doesn't have unit tests. It was built by a hobbiest 4 years ago. The current version supports legacy features from a previous version. It is mainly procedural with a smattering of random classes. There is a lot of code duplication. It is the definition of spaghetti code.

Honestly I wouldn't know where to begin with unit tests in this code base.

Re: Poll: How often do you deploy to production?

#58

Post by mbeattie »

StavrosK wrote: post #44
bendavis381 wrote: post #31

Earlier quoted context omitted.

Have a look at Fabric ( http://docs.fabfile.org/en/1.6/ ) I wrote a quick fabfile yesterday that automatically commits/merges my local changes, pushes to a private GitHub repo, then SSHs into my production server, pulls down the latest code and restarts the web server. Gives me feedback along the way too.

No, don't look at fabric! Fabric is good for some stuff, but deployments should be repeatable, automated and idempotent. Look at ansible.cc instead. Don't use fabric for deployment (even though it's a fantastic tool for many use cases).

I don't understand... how is that a knock on fabric? I have software built into packages and then config templates and some key/value stuff to substitute in. My fabric scripts log in, install a package, put the generated config files, and then restart the server. I can run this over and over again (repeatable) with one command (automated) and it always ends up at the same current version (idempotent).

Re: Poll: How often do you deploy to production?

#59

Post by kisamoto »

Really depends on how much time I've got for my personal projects but here's my current breakdown;

Work - Continuously, can be within minutes:

* Team commits to SVN trunk, tests run + packages built. Manual tests run (can take some time and more fixing). Merge to production branch, tests run + packages built then pushed out to servers.

Personal - Depends on time but few times a month on average:

* I work in Python and do my own DevOps stuff. (Not a Heroku/Appfog/etc fan..) I use SaltStack (saltstack.org) to keep a 'template' that I can apply to a blank Linux Server (AWS|Digital Ocean|Local VirtualBox for dev etc) and it will set it up for me, same layout, every time. Kind of like Puppet/Chef, but in Python. (It's awesome - definitely check it out)

* Commit to BitBucket (for private repos) -> Jenkins pulls code down -> Runs tests, displays coverage etc -> Builds RPMs & Debs -> Salt pushes out to any connected minion (yup - it also has arbitrary command execution).

I'm going to write a blog post on the entire personal setup if anyone is interested.

Also checking out Linux containers (specifically docker.io) to see if I can speed this up.

Re: Poll: How often do you deploy to production?

#60

Post by StavrosK »

mbeattie wrote: post #58
StavrosK wrote: post #44

Earlier quoted context omitted.

No, don't look at fabric! Fabric is good for some stuff, but deployments should be repeatable, automated and idempotent. Look at ansible.cc instead. Don't use fabric for deployment (even though it's a fantastic tool for many use cases).

I don't understand... how is that a knock on fabric? I have software built into packages and then config templates and some key/value stuff to substitute in. My fabric scripts log in, install a package, put the generated config files, and then restart the server. I can run this over and over again (repeatable) with one command (automated) and it always ends up at the same current version (idempotent).

It's not a knock on fabric, the same way using a hammer on screws isn't a knock on hammers. It works sometimes, but you generally want to use the right tool for the job.
Post reply on HN