Any chance of more detail than you are giving in your posting ? This is extremely interesting stuff, I'd really like to know a lot more about what goes in to achieving this.
what do you want to know?
Continuous Deployment at IMVU: Doing the impossible fifty times a day.
41–50 of 76 posts
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#42I think the original article misguided some people. It all looked very simple, update the code and put it in production. That _is a horrible idea_, as some have noted. What's not horrible is having thousands of tests, on dozen of machines, 9 minutes to-live, with selective updating of users, and rollbacks, as this article has explained. The original post was too light on details, I guess. Its intention was not to be…
Honestly I think it's a gradient.
I'm also one of the developers on a hobby project called http://TIGdb.com (Jeff Lindsay is the other, and has written the majority of the website) We don't have a big Continuous Deploy infrastructure, but we also don't have the users and business requirements of IMVU.
We started with the usual, completely manual deploys and hard-to-setup sandboxes, and have been iterating towards a fully automated setup ever since. The entire time we've been doing this, we've been committing and deploying often. Our users are patient, because we're giving them something they can't get elsewhere and we're giving it to them for free. As we do introduce regressions, we'll post-mortem them (probably using the 5 why's technique) and we'll slowly evolve a system to prevent regressions. If the site is a success, we'll have evolved a world class deploy system. If the site never makes it that big then we won't have wasted time on infrastructure. It's classic lean startup thinking (even though TIGdb is really just a hobby project).
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#43Don't be too disappointed if a single submission gets a lukewarm or confused response on HN. The upmods and comments on here are a lot less consistent than what you're used to. ;) Just keep writing. It's really valuable. Also, it's clear to me why your daily routine might sound like science fiction to the median HN reader: A lot of programmers have never seen a system like this. As those of us who were online during…
I definitely wasn't disappointed. In fact it was refreshing; I didn't realize how hard of a problem other people considered Continuous Deployment (and in fact, how little other people had considered it at all).
I also didn't mean to rag on news.yc, on average the responses here are better than the original posts which is an unparalleled level of quality.
I left the technical issues unspecified in my first post on Continuous Deployment, and the comments on that post already had started discussing the path of solutions we ended up building ourselves!
I still have another post to write about this, because we also ship a native windows client. We ship daily prereleases of it, and roughly biweekly full releases (offered to all users). That's close to, but not quite as impressive, as the update system that google chrome uses. Google definitely still has us beat in certain categories.
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#44Earlier quoted context omitted.
These are automated functional tests. It would be interesting to know if the author uses unit test too and how he writes tests in general. Also some code:test ratio metrics would be indicative of how much weight does all this testing add.
I work at IMVU as well, know the author personally, and can tell you that yes, he unit tests too. In fact, like all of us, writes all code test-first. For test strategy in general, you need a whole seminar. There's a good blog on testing over at http://googletesting.blogspot.com/ As for code:test ratio, I'm not sure what metric you want, but using just php files, there are ~1k active test files and ~7k php files tota…
Thanks a lot, this is pretty much what I wanted to know. Looks like there is not too much overhead and the results are amazing.
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#45Earlier quoted context omitted.
Yes, but they might give you a hard time too if you put out something silly before thinking things through.
The obvious solution, of course, is to think things through.
Right, but doing that 50 times per day is more challenging than once, for example.
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#46From what I have read Facebook use a similar method: commit and deploy often and rollback if something messes up. We also use this method on Plurk.com and have done so for about a year. Thought, IMVU's case is pretty extreme :) The major problem is rolling back client side changes (that are located in scripts or CSS). This is pretty costly to rollback, because of browser cache - we solve this by having real versionin…
That makes sure the code is consistent if the user refreshes the webpage or just visits it the first time. But what happens if the user just keeps the AJAXy web-page open for hours (as I do with Gmail for instance)? If you deploy too often and both frontend+backend code are in flux, you're more likely to end up with an inconsistent code state. I guess you could make the frontend code aware of the code version, includ…
Last time we ran into this one, we made sure as much page state as possible was pushed into the fragment portion of the URL (for bookmarkability as much as anything else).
Then when the AJAX stuff saw a version mismatch, it would wait until the user completed any operation that -wasn't- stored in the fragment and put up an "updating, gimme a sec" box, and refresh itself.
It was a hell of a lot of work but -extremely- slick (which I'm allowed to say because it wasn't me who wrote that part ;)
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#47I tell people that we should aim for this sort of automation and they pat me on the head and say, "No, no, that will never do." I think there's an idea that if something goes wrong because you let an automated system do it, it's somehow much worse than if something goes wrong because there was human error. I don't really understand the reasoning.
Exactly. Drew Perttula put it better than I'll be able to: "IMHO, manual testing has only two advantages: it’s the easiest thing to [try to] do; and it has a lovely accountability chain. You can always blame the developer, and non-technical people will easily accept that this is the “inevitable cost of software engineering”."
The thing I'd be really interested in is how you deal with UI changes - I've never found a satisfactory way to test "is this ugly/confusing" other than letting a few users bang on it on a staging server.
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#48Any chance of more detail than you are giving in your posting ? This is extremely interesting stuff, I'd really like to know a lot more about what goes in to achieving this.
what do you want to know?
Everything :)
No, seriously, I'd be much obliged if you could tell what tools go into your setup, how much of it is created in house - and thus unavailable - and how much of it is off the shelf, preferably open source. I'd very much like to spend time on recreating what you've done there.
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#49Earlier quoted context omitted.
Exactly. Drew Perttula put it better than I'll be able to: "IMHO, manual testing has only two advantages: it’s the easiest thing to [try to] do; and it has a lovely accountability chain. You can always blame the developer, and non-technical people will easily accept that this is the “inevitable cost of software engineering”."
The thing I'd be really interested in is how you deal with UI changes - I've never found a satisfactory way to test "is this ugly/confusing" other than letting a few users bang on it on a staging server.
UI is an interesting problem.
The ultimate solution is to have business metrics drive your UI changes, usually in the form of an A/B test. Then you have a clear winner. This A/B would be run separate from the roll out structure (and indeed, we do LOTS of A/B tests).
Sometimes that's not possible, for a new feature or for content without a clear business metric to evaluate for. Either way we often have someone manually test new UI, so that we're not exposing users to something fundamentally broken. We usually do this by using the existing deploy system, but turning the frontend on only for QA users.
In the end, you do what works and is cheap, and that's usually something slightly different for every project.
Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.
#50I was thinking of doing a basic Django + Selenium + Hudson continuous integration how-to blog post, but this blows me out of the water :)
I would love to see that post. The first question most people ask me is "How do I get there?" and I don't have a great place to point and say "start here"
A well written concise introduction to continuous integration / constant testing would be a boon to this community.