Cohort analysis - User retention in a Rails application

At a former Facebook internship, I worked for the insights team. Think Google Analytics for Facebook Pages, Apps, etc.

One of the things I built that term was a tool to visualize user retention, implementing triangle heatmaps, which I believe were invented in-house by Danny Ferante.

The idea here is to exploit the very quick visual pattern matching we're able to do as humans, and turn that into actionable cohort analysis. From the screenshot in the article (http://i.imgur.com/qBbkZv8.png), I think we can agree that it would become unwieldy with a large number of datapoints.

Compare this to http://imgur.com/sOQ4vrm, a screenshot of the triangle heatmap generated for tcreech's Cover Photo Finder Facebook App. The x-axis represents the cohort (the set of users that installed the app on the same day) broken down by day instead of by week like the article (hence patterns are more granular). The y-axis represents number of days after installation. Each datapoint is then coloured to represent the percentage of users that return to the app on (installation date + number of days).

A number of patterns are captured quite easily:

- A vertical pattern is local to a specific cohort. A new promotion or redesigned sign up page often results in this.

- A horizontal pattern is local to a specific vintage. If your app has a trial period that expires after 7 days, then you'll see your retention plummet across all cohorts horizontally at y=7.

- A diagonal pattern is local to a specific date. If your app is down on January 2nd, then there will be a diagonal blue line (0%) across all cohorts.

I wrote up a work term report for the University of Waterloo detailing triangle heatmaps: http://zeroindexed.com/triangle.pdf

Video released by Facebook explaining triangle heatmaps: https://www.facebook.com/video/video.php?v=3707283286197

I'd love to see someone build in cycle plots in to this tool.

My real frustration with Cohort analyses is that the results are always presented in this arcane format that doesn't actually answer any questions.

The question a cohort analysis is supposed to answer is "Is our retention improving" and to get that answer you're supposed eyeball rows & columns of text and make inferences (and then go ahead and speculate some actions).

Here's a better way: http://insideintercom.io/retention-cohorts-and-visualisation...

Oh I understand completely. It's just that I think triangle heatmaps are awesome visualizations that nobody's heard of. They were my entire life for 4 months and I can't stop talking about them given the opportunity :)

I agree that the common presentation of cohort data is clunky. That's why I built this tool with d3js to visualize cohorts as stacked bars:

http://bslatkin.github.com/cohorts/

"Impact maps" are too blurry in my experience; they assume your traffic is normal and predictable. Cycle plots are fun to look at, but in practice the lines are too volatile to make a pretty chart.

This is a cool opensource tool, many thanks for sharing nate. User activation date is the most well-known cohort definition, but actually you can define a cohort out of pretty much any data point you like, provided it makes sense to then run a longitudinal study on those cohorts. Marketing acquisition channel is another good one.

If people are interested in finding out more about cohort analyses, we wrote a set of 7 articles which might be interesting:

http://www.keplarllp.com/blog/2012/04/cohort-analyses-for-di...

And here's the hands-on tutorial for rolling your own cohort analyses in SQL with SnowPlow:

http://snowplowanalytics.com/analytics/customer-analytics/co...