Live data from Hacker News

The Story Behind the Worst Movie on IMDb

fivethirtyeight.com

61–70 of 141 posts

Re: The Story Behind the Worst Movie on IMDb

#61

Maybe a solution could be weighting the vote according to the user history: a user leaving a single vote on a single movie shouldn't be as influential as an user that voted on a wider range of movies over time

Also the range of their votes. If someone only gives 10s or 1s to everything, their vote of 10, should probably have less weight, than someone who distributes their votes more evenly.

This is not correct. Many people dont bother to vote or review unless the movie for them is at an extreme goodness or badness, so the hassle is worth it. I ve watched a lot of so-so or simply good movies, but I only have voted 5-6 times and those got either a 10 or a 1.

Re: The Story Behind the Worst Movie on IMDb

#62
post #13

Earlier quoted context omitted.

Do you have any interesting examples that you've found?

Excuse the plugging of my own wares, but by coincidence I did some analysis the other day of the disparity between User/Critic ratings of movies on RottenTomatoes: http://benjaminlmoore.wordpress.com/2014/05/05/what-are-the-... Example: "Spy Kids" got a 93% fresh rating from critics, but 45% from audience ratings. Code attached should you want to investigate for yourself.

That's really really awesome, thank you for sharing.

Re: The Story Behind the Worst Movie on IMDb

#63
Years ago, when you could download roms from the mame website, I gave a strong up score to the Tron romset.

That tipped the romset into the spotlight - there were some leaderboards for highest recent activity and so on. Other people started downloading the romset and voting on it.

Suddenly this obscure romset was catapulted into most of the lists for "most active" and "best" etc.

I had rated the game honestly. I had fond memories playing the cab for a week on holiday in my youth. But I was surprised that so many other people felt the same, especially on the Mame platform where the game's controls made it tricky.

All my deliberate attempts at voting shennanigans failed miserably. (Although I haven't investigated MTurk or similar yet.)

I wish there was a site like Meatball wiki where people could share their vote-weighting methods.

Re: The Story Behind the Worst Movie on IMDb

#64
"There are currently more than 235,000 films on IMDb, and ... not a single qualified movie besides “Gunday” rates worse than 1.8."

"The next lowest-rated movie on IMDb — 1.8 stars overall ..."

I am not sure what the writer means by a "qualified" movie, but this one does rate less than 1.8: http://www.imdb.com/title/tt2094870/

It has votes from only 195 users as of this writing, though.

Re: The Story Behind the Worst Movie on IMDb

#65
post #50

I feel like there's an assumption that this data is erroneous for this reason - I think that any reason in which a person gives a negative rating toward a film is valid, be it politically motivated or otherwise. FiveThirtyEight has had a rough start so far so I may be more inclined to seek more nuance than I normally would as they've tended to be extraordinarily simplistic (Silver himself notwithstanding) to date.

But most of the people who has given the rating have not seen the movie. They are giving the lowest rating just to put their point forward. This is not a documentary to put the truth on the screen (I am not sure how much of the movie is fiction). I think the author calls data erroneous as the rating does not represent the actual quality of the movie - a thing which IMDB wants to achieve. He doesn't call it erroneous…

> But most of the people who has given the rating have not seen the movie

I'm sure it's possible, but no rating represents the "actual quality of the movie." A rating is based on a subjective amalgam measurement that encompasses a myriad biases, even when watched and with total scrutiny.

Re: The Story Behind the Worst Movie on IMDb

#66
post #49

Earlier quoted context omitted.

IMHO no single website should be used as a definitive source for the quality of peer-reviewed content. Instead, averages should be taken from different sites, compared to the types of movies usually reviewed as high or low on those respective sites. Each site has its own user base, and those users have their own biases. A review from Rotten Tomatoes will vary greatly from those of sites that include only noteworthy c…

I am interested to explore this idea more. In general, I don't trust online reviews for anything because I don't know what 'normal' is or the motivations that people have for voting, as you say here. If I read any restaurant reviews or ratings, it's under the assumption that they're astroturfed. But, IMdB provides the online reviews I trust the most. I know that the dataset is large enough to be reliable. I feel that…

Rotten Tomatoes does contain different information, and different statistics. For example, take Robin Hood: Prince of Thieves.

IMDb gives a 6.9/10 from 119,115 users. That's the only real statistic it gives us; there is no 'Metascore' for this movie. (Personally I think the Oscar nomination should be mentioned next to this score, but it's sort of buried further down the page)

Rotten Tomatoes, on the other hand, shows us several numbers. The Tomatometer is at 50% for "All Critics", with an average rating of 5.7/10 and 52 reviews. The "Top Critics" Tomatometer is at 36%, with an average rating of 5.9/10 and 14 reviews. The Audience rating, however, is 73%, with an average rating of 3.4/5 from 333,273 users.

--

The above example shows how you can sometimes use IMDb's stats of general ratings as a median reference between the audience numbers and critics numbers. But this doesn't always paint the best picture. Let's take another example: the recent (and generally accepted as a flop) 47 Ronin.

Here, IMDb gives us a 6.3/10 rating from 60,849 users. But it also gives a Metascore of 29/100. Yet the number in a gold star in bigger font is just the '6.3'. So even though this movie has a dramatically lower Metascore, they only feature the general user rating from IMDb.

The story is much more dramatic on Rotten Tomatoes. It has a 13% 'All Critics' Tomatometer rating from 72 reviews, with an average rating of 4.1/10. The 'Top Critics' Tomatometer shows 0% with 13 reviews and an average rating of 2.9/10; not one single top critic liked this film, out of 13 critics! Yet the audience rating shows 51% with an average of 3.3/5 with 53,921 ratings.

We can see how IMDb got its '6.3' rating here: both seem to show a middle-of-the-road rating from general audience ratings. But Rotten Tomatoes' critics stats show this movie isn't worth wasting two hours of your life on. The gamble of whether you may like the movie or not becomes more certainly less likely when you take the critics' reviews into consideration.

--

As we can see above, even within each website, sometimes they have not only greatly varying statistics, but sometimes missing or hidden information (Robin Hood not having a Metacritic rating and not prominently displaying the oscar nod). People weigh their options based on the data they have at hand, so the information you give people - along with its context - will change their minds greatly, regardless of the source.

This is why I think it's much more realistic to look at multiple different sites. You need as much information as you can get, and no one site gives you all the relevant data, as it varies from film to film and user community to user community. It would seem you can't just depend on 'the crowd' to give someone an accurate idea of whether they will like a film; a survey would probably be better, but nobody's going to survey every film they watch.

Re: The Story Behind the Worst Movie on IMDb

#67

IMDB ratings are bull. Lots of great movies, often even classics, rate around 6 on IMDB because the general public considers them "boring". The only movies that escape from the IMDB average are a) decent movies that are loved by the masses, b) great movies the masses don't watch (being in black and white or not in English alone pretty much guarantees 2 bonus points) and c) movies everyone agrees on are total crap.

If the general public considers a movie boring, giving it a mediocre rating seems fair. Other members of the general public will then know to avoid it. They'll want to watch movies they like, not movies you like.

What you may want is a personalized rating based on people with the same tastes as you.

Re: The Story Behind the Worst Movie on IMDb

#68

Maybe a solution could be weighting the vote according to the user history: a user leaving a single vote on a single movie shouldn't be as influential as an user that voted on a wider range of movies over time

The solution is recognizing that a 'honest' or 'good' metric is not possible. Thing is, to produce such a list you need to have some function f, which maps the complex opinions of each and every user of the site into an orderable set, usually integers smaller than 5 or 10.

Re: The Story Behind the Worst Movie on IMDb

#69
post #43

538 is using the wrong metrics. For a movie to be truly, truly, sublimely bad, that badness needs to be recognized - people in some cases will want to watch it for its incredible badness. This means that there are going to be a disproportionate number of both 1 and 10 votes. And the winner based on this is - The Room. http://www.imdb.com/title/tt0368226/ratings?ref_=tt_ov_rt

A clear conversation is difficult in the absence of a clear, canonical definition of "bad". It would be very easy for me to get a few friends together and make a 90 minutes improvised movie filmed on my phone. That would be an absolutely terrible movie, but it wouldn't be a very satisfying answer to the question "What's the worst movie?" because it would be seen by very few people and it would be made with no ambition of quality. I could go further and make a movie that was 90 minutes of a black screen. That would be a worse movie but an even less satisfying answer.

Even if you limit the field to movies that have seen a theatrical release, the same principal applies. Some movies are shockingly bad, but are made with few little effort by people who know they are making a bad movie. These seem less interesting, less sad and less funny than very bad movies made by people trying very hard and spending lots of money.

Personally, when rating the badness of work, I judge it against my own assessment of the potential of the work. The potential isn't directly measurable or quantifiable, but it consists of things like the budget, the vision of the creator, the skill and effort of the artists involved in the creation, etc. etc.

So, for example, I think of Star Wars: Episode 1 as being a worse film than a lot of 50s low-budget sci-fi B-movies, even though I might choose to watch it more readily. The B-movies were pumped out without much effort. They're boring and poorly made on a technical level, but they never really had a chance to be any good. Episode 1 was made on a vast budget, drew on extremely rich source material, and was the culmination of unimaginable amounts of time and effort by a large number of hard-working people. For me, that makes the failure of Episode 1 more profound and more egregious.

Post reply on HN