Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Are film critics losing sync with audiences?

stephenfollows.com

Post reply on HN
Print view View original

71–80 of 421 posts

Re: Are film critics losing sync with audiences?

#71

Post by tshaddox »

logicalmonster wrote: post #69
tshaddox wrote: post #63

Earlier quoted context omitted.

It's also possible for critics to have systematically different preferences than movie-goers that don't deserve to be described pejoratively as "out of touch" or "snobby." In fact, that's precisely what I would expect and hope to see from people who are professional critics. I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue (1: McDonal…

> I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue That's a very good point that the role of the critic is to provide their own expertise and opinions on the overall subject. But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right? Many…

> But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right?

Well, no, because I'm pretty sure most film critics aren't trying to directly provide a "yes or no" recommendation to a general movie-going audience. I've read plenty of film criticism that was entertaining and informative despite me having very little shared taste with the writer. And I suspect if you asked film critics to straight up predict what the RottenTomatoes audience score for a film would be, the aggregate of film critics' predictions would tend to be closer to the RT audience score than the RT critic score itself.

Re: Are film critics losing sync with audiences?

#72

Post by wccrawford »

bufferoverflow wrote: post #43
Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

If your narrative was true, we would always see critics scores lower than the audience (assuming statistically significant number of critics reviewing a given movie). But we see so many counterexamples with radically different ratings: Spy kids, Ad Astra, Noah, King Kong (2005), Babe, The Blair Witch Project, Chicken Run, and thousands of others.

It's possible that those movies failed to advertise to the correct audience. If they pulled in people who thought they were getting 1 experience, but ended up with a different experience, the reviews are going to be lower than if they hit the right audience.

Critics, as noted, see everything and so they wouldn't be affected by bad advertising.

I've definitely been wrong, in both directions, about movies based on their advertising.

Re: Are film critics losing sync with audiences?

#73

Post by wccrawford »

klodolph wrote: post #27
ashtonbaker wrote: post #21

Earlier quoted context omitted.

That's a clever way of looking at it. Did you come up with it yourself? If not, I wonder what you think it says about a person that they were exposed to this idea before they interviewed with you, versus after.

IMO it seems obvious to me, but that might be because I took a statistics class in college. (Not “obvious in retrospect”—I’ve had a number of personal discussions with people where we talk about the difference between critic & user scores on Metacritic.)

You also don't have any emotional or financial investment in the situation, though. ;)

Re: Are film critics losing sync with audiences?

#74

Post by toast0 »

logicalmonster wrote: post #69
tshaddox wrote: post #63

Earlier quoted context omitted.

It's also possible for critics to have systematically different preferences than movie-goers that don't deserve to be described pejoratively as "out of touch" or "snobby." In fact, that's precisely what I would expect and hope to see from people who are professional critics. I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue (1: McDonal…

> I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue That's a very good point that the role of the critic is to provide their own expertise and opinions on the overall subject. But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right? Many…

> But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right?

Not really. A professional critic usually writes a longish review describing the item and why they liked or disliked it, as well as the conclusion. If they are good at description, that's useful to me even if I don't share their likes and dislikes. If they hate things I love, then I can look for their negative reviews as places to start. If they love things I hate, their seal of approval is a good sign for me to keep walking. I can trust their recommendations even if I don't follow them, if I think they are fair and honest and have consistent opinions. If their opinions blow with the wind, I probably won't trust their opinions, even if they agree with mine.

I found Anthony Bourdain trustworthy, but I do not like a lot of the dishes that he really did (no thanks to organ meat and blood sausage), so I wouldn't blindly follow his suggestions.

Re: Are film critics losing sync with audiences?

#75

Post by Kranar »

Cederfjard wrote: post #70
Kranar wrote: post #58

Earlier quoted context omitted.

No I did not come up with it myself. It's a common pitfall in statistics known as selection bias [1]. My problem is that if I ask a researcher about selection bias, they have no problem defining it for me and even giving me a textbook example of it. But when I present to a researcher a real life scenario that is highly susceptible to selection bias, they will almost all fail to identify it. It's a problem where many…

Do you agree that without proper research backing it up, you can’t say for sure that this discrepancy is due to selection bias, even if on the surface it looks like a nice fit? If you had said that you’re looking for candidates to raise it as a possible explanation, instead of seemingly firmly asserting that it is the one and only correct answer, you’d probably have less replies arguing with you here (or perhaps if y…

No I don't agree with you. I think my comment is a reasonable statement of the situation. Is it a 100% correct statement that is absolutely categorically true? No, but most people who are not looking to be pedantic and win an Internet argument won't go looking to nitpick the details of my statement.

What most level headed people will do is understand that my statement is about how you can not reliably compare a group of people who are predisposed to enjoy a movie to a group of people who review almost every single movie that's released, regardless of their predisposition towards it. People are welcome to argue that and honestly I'd expect nothing less, but it's a basic statistical principle that I expect every single researcher I hire to be well familiar with and if they're not, then I make the choice to not work with them.

>you’d probably have less replies arguing with you here...

There is also a selection bias among the people who comment on HN and they too are not representative of how most of the community feels towards a topic. The people who want to nitpick the specific wording of my statement are certainly entitled to do so and will likely find something to nitpick regardless of how I framed my statement, but it's not a particularly interesting discussion to follow up on.

Re: Are film critics losing sync with audiences?

#76

Post by yakak »

Kranar wrote: post #75
Cederfjard wrote: post #70

Earlier quoted context omitted.

Do you agree that without proper research backing it up, you can’t say for sure that this discrepancy is due to selection bias, even if on the surface it looks like a nice fit? If you had said that you’re looking for candidates to raise it as a possible explanation, instead of seemingly firmly asserting that it is the one and only correct answer, you’d probably have less replies arguing with you here (or perhaps if y…

No I don't agree with you. I think my comment is a reasonable statement of the situation. Is it a 100% correct statement that is absolutely categorically true? No, but most people who are not looking to be pedantic and win an Internet argument won't go looking to nitpick the details of my statement. What most level headed people will do is understand that my statement is about how you can not reliably compare a group…

Sure, though I think the other implication from the difference stated is that critics have sampled randomly or more often and can therefore recognize derivative work where a target audience that shares a limited interest in film will almost uniformly perceive novelty. I think realizing something is a poor knock off is the basis of snobbery.. so it is strange not to accept that answer.

Re: Are film critics losing sync with audiences?

#77

Post by onlyrealcuzzo »

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

Also - a non-trivial percentage of ratings on Rotten Tomatoes & iMDB (and all review sites) come from accounts that have reviewed more movies than there is time in a human life to watch.

The amount fake user reviews can skew a rating is not to be taken lightly - and IIUC - Rotten Tomatoes & iMDB might be filtering out some/a lot of spam - but definitely not even close to all of it.

The same is true for GoodReads - the amount of teenage booktubers that have reviewed 10k+ books is awfully suspicious...

Re: Are film critics losing sync with audiences?

#78

Post by michaelmrose »

shantnutiwari wrote: post #26

Were they ever in sync? I've always seen critics as a "negative recommendation" -- if a critic recommends a movie, it's usually bad and vice versa (and like all rules there notable exceptions like Thor Ragnarok, which both critics and normal people loved). And this is true not just in movies but books as well. Critics in both domains love the "ooh lala, fancy pancy hard to understand arty farty" book/movie that doesn…

Have you ever considered that there might actually be something to "ooh lala, fancy pancy hard to understand arty farty" movies that you are merely unable to appreciate and that your failure to appreciate any virtue doesn't constitute proof of its nonexistence?

The whole point of pursuing the opinion of critics isn't averaging them to see what smart people like its in finding critics whose taste isn't wholly out of line with yours and using recommendations to cut through the crap and find things worth watching.

Re: Are film critics losing sync with audiences?

#79

Post by pjmlp »

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

As someone that enjoys traveling and testing food places, the same applies to food critics.

Most of the time when I fall again into the trap of following their advices about some place, I come out disappointed, usually getting some creative dishes that still leave me wishing to go eat elsewhere.

I have discovered so many nice places just by randomly crashing into them, where one could watch from outside people were enjoying being there, regardless of how many stars the food critic gave it.

Re: Are film critics losing sync with audiences?

#80

Post by im3w1l »

tshaddox wrote: post #63
logicalmonster wrote: post #24

Earlier quoted context omitted.

> come to the correct conclusion, What makes you so sure that this is the correct interpretation? Both things could be true at the same time: there could be selection bias at the same time as critics are perhaps out of touch and snobby. You could backtest this theory: see if perhaps critics scores were at one time a bit closer to general audiences or not. I think you'd find that at one time good critics were a bit cl…

It's also possible for critics to have systematically different preferences than movie-goers that don't deserve to be described pejoratively as "out of touch" or "snobby." In fact, that's precisely what I would expect and hope to see from people who are professional critics. I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue (1: McDonal…

McDonalds is much cheaper than a fancy restaurant, so it should come as no surprise that it has worse quality. But with movies, people are chosing the lower rated movies even though it doesn't save money.
Post reply on HN