Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Continuous Deployment at IMVU: Doing the impossible fifty times a day.

timothyfitz.wordpress.com

Post reply on HN
Print view View original

31–40 of 76 posts

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#31

Post by delano »

amix wrote: post #19
delano wrote: post #14

Earlier quoted context omitted.

What do you mean by real versioning? The major advantage to using "script.js?v={timestamp}" is that it maintains a consistent URI for the resource. Whereas with "script_{timestamp}.js", everything that points to it needs to be updated every time it changes. You could create a symbolic link or rewrite rule that directs requests for "script.js" to the latest "script_{timestamp}.js" but it's more convenient to use a URI…

The problem with script.js?v={timestamp} is that it's ignored by some browsers while script_{timestamp}.js isn't. And with script.js?v={timestamp} you can't set good cache headers. Also, if you ever move to a CDN, then you are forced to use real versioning (at least with Amazon Cloudfront). The versioning scheme we use is `md5 hash of name + file contents + file extension` (and not timestamp).

I'm not aware of a browser that ignores URI parameters.

Moving to a CDN does not force you to put versioning in the path or filename. The URI parameter merely tricks the browser into thinking there is a new file. The parameter itself is otherwise ignored.

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#33

Post by inerte »

I think the original article misguided some people. It all looked very simple, update the code and put it in production. That _is a horrible idea_, as some have noted.

What's not horrible is having thousands of tests, on dozen of machines, 9 minutes to-live, with selective updating of users, and rollbacks, as this article has explained.

The original post was too light on details, I guess. Its intention was not to be comprehensive anyway, the focus was why recently changed code should be put in production ASAP. But it looked like the author was simply FTPing after commit. And the whole "SOMEONE IS WRONG ON THE INTERNET" thing kicked in.

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#34

Post by lgriffith »

mechanical_fish wrote: post #23
lgriffith wrote: post #12

Interesting idea but.... Looks like meeting that goal would constrain you to write code to be used by a robot and not by a human. There may be many cases where this is both doable and acceptable to the end user. So no problem with that. I am greatly challenged to see how this could be done for a highly interactive, visually oriented, subtle pattern generating response to user input, type application. Computers are st…

As I noted elsewhere in this thread ( http://news.ycombinator.com/item?id=475391 ), this article is not merely about automated tests. The author says that his company is using continuous deployment because it lets live, human end users bang on the code, as quickly as possible, in bite-sized chunks that can more easily be rolled back and fixed.

Then why do such an exhaustive automated test?

Why not have your local tests, automated or not, cover the common cases and error conditions to catch programmer stupidities? Then let the actual humans do the strange corner cases.

If your design is even close to correct, testing repeatedly tested code is pointless. If your design is corrupt and your implementation is sloppy, no amount of testing is going to save your ass.

I do very rapid turns and I am a one man team. I can turn my system in less that 30 minutes and have the user testing it in a live situation on the other coast. If I want 10 turns a day, I can easily do it. Low coupling, high cohesion, clean correct design, and disciplined implementation makes it possible.

I agree that doing things in small chunks is a great way to do it but doing the equivalent of a weeks worth of global automated testing for each small change seems like a silly exercise. That is except for the server hardware salesmen and system admin people.

The sales commissions and payroll look rather good. The production of real value is questionable. Bang for the buck is as important for testing as it is in any other part of product development.

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#35

Post by shiranaihito »

teej wrote: post #32
shiranaihito wrote: post #25

Why do all changes have to end up in production immediately?

Because having thousands of real users running your code gives you insight that automate tests simply cannot match.

Yes, but they might give you a hard time too if you put out something silly before thinking things through.

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#36

Post by amix »

delano wrote: post #31
amix wrote: post #19

Earlier quoted context omitted.

The problem with script.js?v={timestamp} is that it's ignored by some browsers while script_{timestamp}.js isn't. And with script.js?v={timestamp} you can't set good cache headers. Also, if you ever move to a CDN, then you are forced to use real versioning (at least with Amazon Cloudfront). The versioning scheme we use is `md5 hash of name + file contents + file extension` (and not timestamp).

I'm not aware of a browser that ignores URI parameters. Moving to a CDN does not force you to put versioning in the path or filename. The URI parameter merely tricks the browser into thinking there is a new file. The parameter itself is otherwise ignored.

Amazon Cloudfront forces you to put versioning if you want to expire objects manually, check out this page: http://docs.amazonwebservices.com/AmazonCloudFront/2008-06-3... (under `Object expiration`).

Unless you specify "Cache-control: no-cache" header you aren't really sure how the browser caches your static files (especially if the user is behind a proxy - and even "Cache-control: no-cache" can easily be ignored).

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#37

Post by jmathes »

ntoshev wrote: post #5
patio11 wrote: post #2

One of the greatest lines I have ever read on a blog: It may be hard to imagine writing rock solid one-in-a-million-or-better tests that drive Internet Explorer to click ajax frontend buttons executing backend apache, php, memcache, mysql, java and solr. I am writing this blog post to tell you that not only is it possible, it’s just one part of my day job.

These are automated functional tests. It would be interesting to know if the author uses unit test too and how he writes tests in general. Also some code:test ratio metrics would be indicative of how much weight does all this testing add.

I work at IMVU as well, know the author personally, and can tell you that yes, he unit tests too. In fact, like all of us, writes all code test-first.

For test strategy in general, you need a whole seminar. There's a good blog on testing over at http://googletesting.blogspot.com/ As for code:test ratio, I'm not sure what metric you want, but using just php files, there are ~1k active test files and ~7k php files total. So if you take into account that 1k of that 7k is the test, and some is test-setup and obsolete files we no longer use, it's maybe about 5:1 code:test. However, a lot of the code is third party OS software packages. In-house code is about 1.5:1 test:code, in my experience. These, by the way, are very rough estimates.

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#38

Post by jmathes »

lgriffith wrote: post #34
mechanical_fish wrote: post #23

Earlier quoted context omitted.

As I noted elsewhere in this thread ( http://news.ycombinator.com/item?id=475391 ), this article is not merely about automated tests. The author says that his company is using continuous deployment because it lets live, human end users bang on the code, as quickly as possible, in bite-sized chunks that can more easily be rolled back and fixed.

Then why do such an exhaustive automated test? Why not have your local tests, automated or not, cover the common cases and error conditions to catch programmer stupidities? Then let the actual humans do the strange corner cases. If your design is even close to correct, testing repeatedly tested code is pointless. If your design is corrupt and your implementation is sloppy, no amount of testing is going to save your a…

I think the source of your confusion is that you're a one man team. You don't have to solve problems that 20-man teams have to face. At least half of all the code I depend on is code I do not understand, so I have to depend on its tests, and I have to make the same promise to consumers of my code. If my change breaks code someone else wrote that I didn't foresee, I am depending on his tests to tell me what I screwed up.

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#39

Post by BeefingJection »

shiranaihito wrote: post #35
teej wrote: post #32

Earlier quoted context omitted.

Because having thousands of real users running your code gives you insight that automate tests simply cannot match.

Yes, but they might give you a hard time too if you put out something silly before thinking things through.

The obvious solution, of course, is to think things through.

Re: Continuous Deployment at IMVU: Doing the impossible fifty times a day.

#40

Post by TimothyFitz »

DannoHung wrote: post #4

I tell people that we should aim for this sort of automation and they pat me on the head and say, "No, no, that will never do." I think there's an idea that if something goes wrong because you let an automated system do it, it's somehow much worse than if something goes wrong because there was human error. I don't really understand the reasoning.

Exactly. Drew Perttula put it better than I'll be able to: "IMHO, manual testing has only two advantages: it’s the easiest thing to [try to] do; and it has a lovely accountability chain. You can always blame the developer, and non-technical people will easily accept that this is the “inevitable cost of software engineering”."
Post reply on HN