Live data from Hacker News

Are film critics losing sync with audiences?

stephenfollows.com

61–70 of 421 posts

Re: Are film critics losing sync with audiences?

#61
I suspect film might just be dead as an artistic medium that can be enjoyed by lots of people. I maybe go to a movie theatre once every couple of years these days, the rest of the time I watch things at home. It’s a hard ask to set aside two contiguous hours (or more!) to watch a film when there are so many good series or video games that demand less contiguous attention. People will rant about attention spans I guess but it’s really a matter of what quality of entertainment you can fit in with the rest of your life. If I watch a film in the evening that is pretty much the only thing I do with that evening; not so with other entertainment formats. The two hour film is also like the equivalent of a short story: there’s the mental barrier to entry of loading new characters and settings and world rules into your head, and so I find it more difficult to read anthologies of short stories than a novel of the same length.

Re: Are film critics losing sync with audiences?

#62
post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

Calling that the "correct conclusion" is absurd. Whether or not a critic "wanted" to see a movie shouldn't really factor into the review at all. At the end of the day, the typical movie-goer is probably judging a film based off of an entirely different set of criteria (excitement / satisfaction) than a typical movie critic (overall artistic merit). Which isn't an exciting conclusion, but probably more correct.

I would personally rate many movies I've seen lower if you told me to judge it from the perspective of a movie critic as opposed to a general audience member (The Purge for example).

Re: Are film critics losing sync with audiences?

#63
post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

> come to the correct conclusion, What makes you so sure that this is the correct interpretation? Both things could be true at the same time: there could be selection bias at the same time as critics are perhaps out of touch and snobby. You could backtest this theory: see if perhaps critics scores were at one time a bit closer to general audiences or not. I think you'd find that at one time good critics were a bit cl…

It's also possible for critics to have systematically different preferences than movie-goers that don't deserve to be described pejoratively as "out of touch" or "snobby." In fact, that's precisely what I would expect and hope to see from people who are professional critics. I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue (1: McDonalds, 2: Starbucks, etc.).

Re: Are film critics losing sync with audiences?

#65
post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

I used to be a film critic for the largest newspaper in a pretty big US city. Movie critics absolutely do not watch every movie. Not even close.

Fair point, it's categorically true that no critic watches every single movie released and I definitely overstated that. A brief search on Statista indicates that about 700 movies are released a year, the average American sees 2.3 movies a year (in theaters), and professional critics review over 200 movies a year, some even review 300.

I maintain that given this disparity (2.3 movies a year versus 200 a year) my point still stands, critics will tend to watch movies regardless of any predisposition towards them whereas an audience member will be much more discriminating about which 2.3 movies they will pay to see.

Re: Are film critics losing sync with audiences?

#66

Earlier quoted context omitted.

Gotcha type interview questions with a single "trick" that you either get/have seen before or don't provide terrible signal imo.

The comment literally states that it's one of the questions they ask, which is the exact opposite of a gotcha question. And the specific question can lead to a conversation, which may "reveal" the answer (obviously, I don't know whether this is the correct answer), but at the very least it reveals whether the interviewee thinks about selection bias, among other concerns.

> The comment literally states that it's one of the questions they ask, which is the exact opposite of a gotcha question.

Huh? "Gotcha question" doesn't imply it's a single question that's not part of a series of questions. A "gotcha question" is simply a question designed such that it's very easy to respond with something that sounds bad or is incorrect.

Re: Are film critics losing sync with audiences?

#67
post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

> audience typically only goes to see movies that they are likely to enjoy

That doesn't really explain the opposite phenomenon: movies that have higher critic rating than audience rating (e.g. movies like Mulan and Turning Red come to mind). IMHO, if you were to apply that rubric across all movies, the conclusion you should get is that critics and general audiences are simply different demographics and that it is normal for discrepancies to arise from sampling different demographics, not that general audiences are more biased towards what they like to watch. For example, parents wouldn't necessarily pick kids movies as their first choice of entertainment if age appropriate-ness wasn't a consideration (I think a lot of parents can relate to being sick of replaying Frozen, for example)

Re: Are film critics losing sync with audiences?

#68
post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

"The score, made up of audience ratings on a scale of 100, is calculated by taking the percentage of people who rated it at least 60 out of 100 (or 6 out 10) and multiplying it by 100%." The critics score is calculated with a different formula. Do you discuss this with candidates? Maybe research your interview questions a bit more before using them.

I absolutely do go over that, I also go over Metacritic and point out that this phenomenon applies to movies, video games, music, books, and almost all review systems that segregate professional reviews with customer reviews.

Re: Are film critics losing sync with audiences?

#69

Earlier quoted context omitted.

> come to the correct conclusion, What makes you so sure that this is the correct interpretation? Both things could be true at the same time: there could be selection bias at the same time as critics are perhaps out of touch and snobby. You could backtest this theory: see if perhaps critics scores were at one time a bit closer to general audiences or not. I think you'd find that at one time good critics were a bit cl…

It's also possible for critics to have systematically different preferences than movie-goers that don't deserve to be described pejoratively as "out of touch" or "snobby." In fact, that's precisely what I would expect and hope to see from people who are professional critics. I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue (1: McDonal…

> I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue

That's a very good point that the role of the critic is to provide their own expertise and opinions on the overall subject. But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right?

Many Americans might listen to a food critic that said that he thought some pasta dish at some Italian restaurant was the best, even if pasta is not currently everybody's favorite food. But if he he recommended boiled snails or braised calf brains, many people might ignore his judgement because it's a bit too far out of range of what they're comfortable with. At a certain distance from popular opinion, a mass audience might no longer trust that critic. Just a thought.

Re: Are film critics losing sync with audiences?

#70
post #58

Earlier quoted context omitted.

That's a clever way of looking at it. Did you come up with it yourself? If not, I wonder what you think it says about a person that they were exposed to this idea before they interviewed with you, versus after.

No I did not come up with it myself. It's a common pitfall in statistics known as selection bias [1]. My problem is that if I ask a researcher about selection bias, they have no problem defining it for me and even giving me a textbook example of it. But when I present to a researcher a real life scenario that is highly susceptible to selection bias, they will almost all fail to identify it. It's a problem where many…

Do you agree that without proper research backing it up, you can’t say for sure that this discrepancy is due to selection bias, even if on the surface it looks like a nice fit?

If you had said that you’re looking for candidates to raise it as a possible explanation, instead of seemingly firmly asserting that it is the one and only correct answer, you’d probably have less replies arguing with you here (or perhaps if you do have such research readily available, to have shared it).

Post reply on HN