Are film critics losing sync with audiences? • Hacker News Board

Re: Are film critics losing sync with audiences?

#51

Post by UnpossibleJim » Tue, Apr 12, 2022, 5:12 PM UTC

bufferoverflow wrote: post #43

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

If your narrative was true, we would always see critics scores lower than the audience (assuming statistically significant number of critics reviewing a given movie). But we see so many counterexamples with radically different ratings: Spy kids, Ad Astra, Noah, King Kong (2005), Babe, The Blair Witch Project, Chicken Run, and thousands of others.

Not to mention a number of films who stopped the audience scores, decrying "review bombing". Some of these films were fine, while some were objectively bad (-cough- Ghostbusters reboot -cough-). I have a feeling (though it is unsubstantiated rumor) that there are payoffs in the critics circles for certain movies.

Re: Are film critics losing sync with audiences?

#52

Post by photochemsyn » Tue, Apr 12, 2022, 5:12 PM UTC

Another possibilty is that audience scores are easier to gamify via the use of bots relative to critic scores (which I assume are limited to identified critics who work for major media outlets). One way for a movie's PR/advertising group to generate buzz is to flood rt with positive movie reviews for example.

I suppose you could detect this by correlating individual movie advertising budgets with rt audience scores across the board?

Re: Are film critics losing sync with audiences?

#53

Post by pie42000 » Tue, Apr 12, 2022, 5:12 PM UTC

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

"A very small minority of UX Researchers come to the correct conclusion, which is that most professional critics have to review every single movie"

Could you provide your sources? This seems like it could be very incorrect, or only partially correct. Keep in mind rottentomatoes is owned by movie studios so they have a vested interest in changing the formula for disregarding audience ratings. I think they already disregard ratings below a certain threshold and other weird shit.

Re: Are film critics losing sync with audiences?

#54

Post by thaway2839 » Tue, Apr 12, 2022, 5:13 PM UTC

fullshark wrote: post #31

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

Gotcha type interview questions with a single "trick" that you either get/have seen before or don't provide terrible signal imo.

The comment literally states that it's one of the questions they ask, which is the exact opposite of a gotcha question.

And the specific question can lead to a conversation, which may "reveal" the answer (obviously, I don't know whether this is the correct answer), but at the very least it reveals whether the interviewee thinks about selection bias, among other concerns.

Re: Are film critics losing sync with audiences?

#55

Post by qiskit » Tue, Apr 12, 2022, 5:14 PM UTC

logicalmonster wrote: post #17

I don't know if this tells the entire story, but I think these are 2 of the bigger issues at play. 1) The corporatization of social media is a big factor. There's big money involved in controlling reviews. Critics and influencers devoted to certain fandoms inevitably get influenced, pressured, infiltrated by studios who expect favorable treatment towards their product or that entity loses access, gets attacked, etc.…

> 1) The corporatization of social media is a big factor.

Yep. There was a time when I trusted internet/social media reviews because they weren't bought and paid for access media. But now, it's clear that they are also bought and paid for. They are now invited to press screenings and follow disney rules on release of spoiler reviews. Not only influencers but entire social media platforms are now arms of corporate PR.

> 2) Everything has become political

It's always been political. The only difference now is that ordinary people have a way of voicing their opinions. Hollywood, media, film critics, best sellers list, award shows, etc are all political propaganda. Always have been.

Re: Are film critics losing sync with audiences?

#56

Post by nix0n » Tue, Apr 12, 2022, 5:14 PM UTC

One possibly overlooked factor is the death of Roger Ebert, who managed to be both a man of the people and respected among critics (possibly because he was also an excellent writer).

Re: Are film critics losing sync with audiences?

#57

Post by davidktr » Tue, Apr 12, 2022, 5:16 PM UTC

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

Then how do you explain that there are systematic differences for certain film genres? The article has a graphic on it. The differences are by no means uniform. Crowds love Action and Thriller, critiques History and Documentary. D'oh.

That is not to say there is no selection bias. But the population of critiques and film-going crowds are almost certainly different in education, lifestyle etc. That has certainly an effect on the observed discrepancies.

Re: Are film critics losing sync with audiences?

#58

Post by Kranar » Tue, Apr 12, 2022, 5:17 PM UTC

ashtonbaker wrote: post #21

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

That's a clever way of looking at it. Did you come up with it yourself? If not, I wonder what you think it says about a person that they were exposed to this idea before they interviewed with you, versus after.

No I did not come up with it myself. It's a common pitfall in statistics known as selection bias [1].

My problem is that if I ask a researcher about selection bias, they have no problem defining it for me and even giving me a textbook example of it. But when I present to a researcher a real life scenario that is highly susceptible to selection bias, they will almost all fail to identify it. It's a problem where many people are able to compartmentalize knowledge but fail to actually make use of that knowledge in practice.

As a run a quant firm I see this among so many researchers across many disciplines. They fully understand a concept when you ask them directly, but they fail to put those concepts to use outside of a very direct and artificial setting.

[1] https://en.wikipedia.org/wiki/Selection_bias

Re: Are film critics losing sync with audiences?

#59

Post by pie42000 » Tue, Apr 12, 2022, 5:18 PM UTC

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

"The score, made up of audience ratings on a scale of 100, is calculated by taking the percentage of people who rated it at least 60 out of 100 (or 6 out 10) and multiplying it by 100%."

The critics score is calculated with a different formula. Do you discuss this with candidates?

Maybe research your interview questions a bit more before using them.

Re: Are film critics losing sync with audiences?

#60

Post by rurban » Tue, Apr 12, 2022, 5:21 PM UTC

nopenopenopeno wrote: post #44

Kranar wrote: post #8

One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case. The overwhelming majority of answers involve critics being out of touch with the audience, critics being per…

I used to be a film critic for the largest newspaper in a pretty big US city. Movie critics absolutely do not watch every movie. Not even close.

I used to be a better professional film critic, mostly festival movies.

movie critics do watch the best movies. eg on a festival with 100-250 movies they watch from 30 to 50 typically. from these 30-50 there are about 20 good movies, the best movies of the year. from these regular moviegoers watch 1-2. normal people do not watch the best but the worst with the biggest ad budget.

of course critics don't watch every trash movie which starts every week. from 5-8 filmstarts per week, they watch 1-3. these weeks more, around 3, because now we have the best weeks of the year, with last year's Cannes movies arriving. and then again late fall, with the best Oscar movies being pushed. but eg this year's Oscars had only 2 good movies overall.

for comparison: my letterboxd https://letterboxd.com/rurban/films/diary/