Do film critics punish films with bigger budgets?

28 September '20 13 Comments on Do film critics punish films with bigger budgets?

Last week, a reader reached out to ask a question around film reviews. They were discussing the critical reaction to Tenet with friends and one suggested that many of the poor reviews were unreliable as they were the typical reaction snotty film critics have towards big-budget movies. The reader asked what the data reveals on this topic.

It’s worth noting that Tenet has received a mostly favourable reaction from critics, although it is true that some of the less-positive reviews have commented on the scale of the film.

  • If you just want big, then “Tenet” is as big as the world, a scale Nolan flaunts by traversing the planet twice, in different directions… An insinuating mid-budget noir has been punched up into a bet-the-house studio actionerIndieWire
  • [Christopher Nolan has] …devoted so much of Tenet to the Bond-alike sequences that the later science-fiction sequences are frustratingly hurried, undeveloped and almost impossible to make sense of.” BBC
  • Maximally staged and very, very loud, but flimsy at its heartVanity Fair

But the quality or budget of Tenet is not our focus. We are looking at whether film critics typically give poorer reviews to bigger budget movies.

I gathered data on 7,224 movies which were released in US cinemas between 2000 and 2019, and focused on the scores they received from film critics (a total of 159,486 individual reviews). I then looked for the correlation between the review scores and the film’s budget, where a figure was available.

Let’s start by answering the question directly, and then break it down by top publications and critics.

What is the overall connection between film reviews and movie budgets?

We can measure the correlation between two sets of figures by using the Pearson coefficient. It gives us a score between minus one and one. Minus ones means they are perfectly negatively correlated (i.e. whenever one goes up the other goes down), a score of one means perfect positive correlation (i.e. they rise and fall together) and zero means no correlation.

Across the whole dataset, there is no connection between budget and review scores (with a Pearson score of -0.07, which is not statistically significant). That said, if there were, it would only reveal the connection, not prove bias. It could be that bigger budget movies are actually better/worse than smaller titles.

Where it gets interesting is to look at which individual publications and critics are relatively big-budget-phobic when compared to their peers.

Which publications are harshest to big budget movies?

The Wall Street Journal is the harshest to big budget movies, within the thirty top publications I studied, followed closely by IndieWire and The New Yorker.

No publications favour big budget movies over smaller ones.

Which film critics are big-budget-phobic?

The chart below shows the scores for the 103 reviewers I analysed. I have left off most names as (a) there isn’t space; and (b) many fall below the level of statistical significance to be useful (the scores, not the critics!)

The reviewers whose scores are most critical of big budget movies are:

  • Stephanie Merry (score of -0.35) who writes for the Washington Post. Of Exodus: Gods and Monsters she wrote “This biblical action drama that feels excessive in every way imaginable, from running time (nearly 2 1/2  hours) to melodramatic acting to the conspicuous amount of computer generation“.
  • Emily Yoshida (score of -0.35) who writes for New York Magazine. She described The Meg as “a more or less straight-ahead B-movie thriller, with a heavy injection of Chinese financing and a rather shocking thirst for bloody whale carcasses“.
  • Kenneth Turan (score of -0.26) at the Los Angeles Times. His review of Gangs of New York included the line “Scorsese and his team have created a heavy-footed golem of a motion picture, hard to ignore as it throws its weight around but fatally lacking in anything resembling soul“.
  • David Sims (score of -0.34) at The Atlantic. Of Hobbs and Shaw he wrote “[Jason Statham and The Rock] spend most of the movie yelling at each other, preening over who’s the better action hero and threatening to shove various household objects into various orifices, and only really come together for the final 20 minutes (long after I had zoned out)“.
  • Katie Rife (score of -0.31) at The A.V. Club. She neatly summed up why she didn’t like Godzilla: King of the Monsters with the line “you can’t just have two hours of kaiju slapping each other around like a gargantuan WWE highlights reel“.

The correlation at the other end of the spectrum (i.e. favouring big-budget movies over smaller ones) was weaker. Tom Russo at the Boston Globe had the strongest positive correlation (score of +0.16). In his glowing review of Ant-Man and The Wasp he said “It’s as close to lizard-brain entertainment as superhero blockbusters get“.

Notes

The data today came from IMDb, Wikipedia and from the film reviews, via Metacritic. I focused on publications which had published at least 300 movie reviews in the past five years and for which I have budget data. For reviewers that threshold was 100 reviews. Once I had my chosen publications and reviewers, I studied all their movie reviews since 2000.

The quotes are to get a sense of the critics’ writing styles and for colour. The point is the data, not the anecdotal snippets I selected.

Epilogue

A few years ago, I was disappointed to see a reviewer for a major publication receiving public taunting for not liking Deadpool 2. She had tweeted ahead of the film that she was dreading the upcoming press screening and was probably not going to like the film. Using screenshots of her tweet and her resulting poor review, online comments flooded in complaining that she was biased and the review should be taken down.

It could be argued that she was the wrong journalist to be assigned to the role, although even that is debatable. Should movies only be reviewed by people who expect to like the movie? In either case, she was not at fault for disliking a movie she wouldn’t choose to watch in her personal life. I’m sure each critic has their own experiences like this and I only single this one case out as it has stuck with me. The anger and cries for retribution were way out of whack with someone not liking a movie.

By its very nature, film criticism is subjective because (a) art can’t have a single objective verdict; and (b) that’s the point – you want to learn someone’s perspective. To me, this seems self-evident. And yet, so many of the public comments added to film reviews and on Twitter are based on the premise that film reviewers could be reaching a single universal objective truth if they weren’t so damned lazy / biased / female.

Share

Comments

13 comments
  1. I like this.
    At the risk of sounding like a misogynistic Twitter troll who gets in a flap about female critics, is it significant that four of the top five most anti-big-budget critics (as shown in your bar chart) are women and that all five pro-big-budget critics are men?
    This could just be coincidence. Adjusting for the overall proportion of male/female critics, are there a larger numbers of women on the higher reaches of your chart?

    1. I don’t even know why Alice is pulled out. She’s below a correlation line for Pearson’s (.3 is the limit for relevance, IIRC), and the detail of the chart makes her appear far closer to the 4 (barely) correlating reviewers instead of say, the 6 others with similar Pearson’s.

  2. Stephen…
    This one of the most intriguing and insightful of your always interesting analyses that I’ve seen. I am in the final stages of publishing a book about movies and their impact on our culture. I try to address this very question in the book. In fact, I have a whole chapter on critics and the role they play. You are on my short list of people who I plan on sending a complimentary copy (look for it in November or December. Please continue with your very astute and very relevant articles. You make an important contribution to the world of film enthusiasts.

  3. I’m not a statistician, so apologies if I’m missing something, but it seems to me we need to know a critic’s score in relation to ALL movies reviewed if we want to know whether they have a bias for or against big budget productions. In other words, if Critic A scores -0.25 on big budget movies–which would place them pretty high on the negative scale–but has a -0.30 score for their full range of reviews, then that would indicate they may actually favor larger budgets. Some critics just produce more negative reviews than others, and vice versa. Wouldn’t that skew these results?

    1. Hi Michael. This is looking at a measure of their correlation, not an average of their score. So it doesn’t actually matter whether they always give good or bad reviews, but rather the level to which the rise or fall in line with the films’ budgets.

  4. Having seen Tenet it *is* pretty flimsy. It is big budget but as a viewer it didn’t appear any bigger than, say, the recent Mission Impossibles.

    I’d say all effort was focused on the effects more so than the story. The Protagonist lacks a name (even his title seems like a holding name) and no character arc compared with say, Inception.

  5. A great reminder that critics are humans and have their own taste. It is also a reminder to those making lower budget films that they have a chance to gain an audience through good reviews, and a reminder to big budget films that a big budget does not mean that they can throw out the elements of a tasteful, good story.

  6. I don’t think you can do this kind of analysis. Some critics work for a local newspaper and only cover “the movie of the week”, that is, the expensive movie that is opening and cost hundreds of millions to produce. Obviously, all of their “punishment” will go to big budget movies. Not because they dislike them in particular, but because these are the only movies they write about.

    Conversely, some critics only cover festivals and “artsy movies”. Their punishment will only go to small budget movies. Because they don’t talk about the newest Avengers flick, their space is dedicated to Sundance and Toronto film festival…

  7. Thank you for this amazing info. If you can, please write some article about how much cost to rent the cinematic equipment like Arri Alexa 65 or Arri Alexa LF (or Arri Alexa mini LF) and so on. Or, more specifically, how are the budget of a movie distributed in percentage over pre-production, filming, post-production, promoting etc. I’m interested to know where the most of the money go, to think if there can be optimization and to buy a RED camera instead of renting an old 2013 Arri Alexa XT and lenses.

  8. Lovely correlation critic x movie budget Stephen ! It seems to me that the role of the movie critic has shifted from a curator to a validator somewhere in the last forty years…
    I remember reading movie critics after the movie and get a lot of insights about story, context, creative choices and market… but in those recent decades, those journalists try to reveal the plot and analyse in an academic way critically if the movie has matched a specific cultural movement configuration… Critics went from writing a paralel complementary reading to enrich the movie experience to become a noisy reduction fighting for atention.
    ps. there is a heartfelt comment on critics in ‘Caro Diario’ from Nanni Moretti.

Leave a Reply

Your email address will not be published. Required fields are marked *

Stephen Follows