A data science investigation of IMDB, Rotten Tomatoes, Metacritic and Fandango

I have many problems with this analysis, perhaps enough to write my own blog post, but I am lazy so I will just outline my disagreements here (at least for now).

First: I claim that what we seek in a movie rating is information about whether we will like the movie, and that this can be formalized as the expected KL-divergence (information gain) between the Bayesian posterior distribution (probability of enjoying the movie conditional on its rating) and the prior distribution (probability you would enjoy a randomly selected movie). Of course, this will depend on your taste in movies, especially how much it correlates with others. But, we can _bound_ it by taking the Shannon entropy of the rating distribution: there is no way we can get more information from a rating than this! It is this bound that allows us to penalize the distributions that are heavily biased towards one side of a discrete scale, like Fandango. However, the "ideal" shape in this context is far from a Gaussian - it is uniform! The uniform distribution can also be justified as being calibrated such that the quantile function is linear - a score of 90/100 from a uniform distribution means a 90th-%ile movie. Determining a quantile is often a transform we try to perform intuitively on ratings so such a transform being trivial seems useful.

Second: The Gaussian distribution does not have bounded support! That is, a rating scheme with what you claim as the "ideal" distribution would have _some_ ratings with values that are negative or otherwise "off the scale". Not so ideal! If you wanted to model movie-goodness on an unbounded scale such that a Gaussian would have sense, then you should transform that scale into a bounded scale, eg with a logistic function, yielding an "ideal" shape of a logitnormal distribution, which incidentally can fit the strange bimodal Tomatometer distribution quite well. Even if you specifically wanted a unimodal, bell-shaped distribution, at least pick a bounded one like the beta distribution.

Third: setting aside which distribution you want to penalize distance from or why, dividing the space into three arbitrary intervals to facilitate the comparison seems ridiculous. There is already a perfectly good metric on probability distributions, the mutual information.

https://www.cinesift.com/ lets you sort by combined rating.

source: https://news.ycombinator.com/item?id=14964701

I was disappointed by the 'data science' in this article. From selecting distribution as a criterion for the qualitry of a metric, for using Pearson correlation with non normal data, for using non correlation with Fandango as tie-breaking criterion, for failing to use external criteria (e.g. ticket sales) to validate or compare metrics, and to be picky, for failing to discuss (for example) generalized lin models with link functions to deal with non normal error distributions.

"Last few years" and "Crystal Skull" are fairly contradictory, so what do you claim has deteriorated recently?

Thanks for the tip. Is there any media recommendation engine that implements it?

Note though that I'm not talking about the engine itself recommending movies. It places you within a group and shows you the scoring data of like-minded reviewers.