Do film critics punish films with bigger budgets?

Last week, a reader reached out to ask a question around film reviews. They were discussing the critical reaction to Tenet with friends and one suggested that many of the poor reviews were unreliable as they were the typical reaction snotty film critics have towards big-budget movies. The reader asked what the data reveals on this topic.

It’s worth noting that Tenet has received a mostly favourable reaction from critics, although it is true that some of the less-positive reviews have commented on the scale of the film.

  • If you just want big, then “Tenet” is as big as the world, a scale Nolan flaunts by traversing the planet twice, in different directions… An insinuating mid-budget noir has been punched up into a bet-the-house studio actionerIndieWire
  • [Christopher Nolan has] …devoted so much of Tenet to the Bond-alike sequences that the later science-fiction sequences are frustratingly hurried, undeveloped and almost impossible to make sense of.” BBC
  • Maximally staged and very, very loud, but flimsy at its heartVanity Fair

But the quality or budget of Tenet is not our focus. We are looking at whether film critics typically give poorer reviews to bigger budget movies.

I gathered data on 7,224 movies which were released in US cinemas between 2000 and 2019, and focused on the scores they received from film critics (a total of 159,486 individual reviews). I then looked for the correlation between the review scores and the film’s budget, where a figure was available.

Let’s start by answering the question directly, and then break it down by top publications and critics.

What is the overall connection between film reviews and movie budgets?

We can measure the correlation between two sets of figures by using the Pearson coefficient. It gives us a score between minus one and one. Minus ones means they are perfectly negatively correlated (i.e. whenever one goes up the other goes down), a score of one means perfect positive correlation (i.e. they rise and fall together) and zero means no correlation.

Across the whole dataset, there is no connection between budget and review scores (with a Pearson score of -0.07, which is not statistically significant). That said, if there were, it would only reveal the connection, not prove bias. It could be that bigger budget movies are actually better/worse than smaller titles.

Where it gets interesting is to look at which individual publications and critics are relatively big-budget-phobic when compared to their peers.

Which publications are harshest to big budget movies?

The Wall Street Journal is the harshest to big budget movies, within the thirty top publications I studied, followed closely by IndieWire and The New Yorker.

No publications favour big budget movies over smaller ones.

Which film critics are big-budget-phobic?

The chart below shows the scores for the 103 reviewers I analysed. I have left off most names as (a) there isn’t space; and (b) many fall below the level of statistical significance to be useful (the scores, not the critics!)

The reviewers whose scores are most critical of big budget movies are:

  • Stephanie Merry (score of -0.35) who writes for the Washington Post. Of Exodus: Gods and Monsters she wrote “This biblical action drama that feels excessive in every way imaginable, from running time (nearly 2 1/2  hours) to melodramatic acting to the conspicuous amount of computer generation“.
  • Emily Yoshida (score of -0.35) who writes for New York Magazine. She described The Meg as “a more or less straight-ahead B-movie thriller, with a heavy injection of Chinese financing and a rather shocking thirst for bloody whale carcasses“.
  • Kenneth Turan (score of -0.26) at the Los Angeles Times. His review of Gangs of New York included the line “Scorsese and his team have created a heavy-footed golem of a motion picture, hard to ignore as it throws its weight around but fatally lacking in anything resembling soul“.
  • David Sims (score of -0.34) at The Atlantic. Of Hobbs and Shaw he wrote “[Jason Statham and The Rock] spend most of the movie yelling at each other, preening over who’s the better action hero and threatening to shove various household objects into various orifices, and only really come together for the final 20 minutes (long after I had zoned out)“.
  • Katie Rife (score of -0.31) at The A.V. Club. She neatly summed up why she didn’t like Godzilla: King of the Monsters with the line “you can’t just have two hours of kaiju slapping each other around like a gargantuan WWE highlights reel“.

The correlation at the other end of the spectrum (i.e. favouring big-budget movies over smaller ones) was weaker. Tom Russo at the Boston Globe had the strongest positive correlation (score of +0.16). In his glowing review of Ant-Man and The Wasp he said “It’s as close to lizard-brain entertainment as superhero blockbusters get“.

Notes

The data today came from IMDb, Wikipedia and from the film reviews, via Metacritic. I focused on publications which had published at least 300 movie reviews in the past five years and for which I have budget data. For reviewers that threshold was 100 reviews. Once I had my chosen publications and reviewers, I studied all their movie reviews since 2000.

The quotes are to get a sense of the critics’ writing styles and for colour. The point is the data, not the anecdotal snippets I selected.

Epilogue

A few years ago, I was disappointed to see a reviewer for a major publication receiving public taunting for not liking Deadpool 2. She had tweeted ahead of the film that she was dreading the upcoming press screening and was probably not going to like the film. Using screenshots of her tweet and her resulting poor review, online comments flooded in complaining that she was biased and the review should be taken down.

It could be argued that she was the wrong journalist to be assigned to the role, although even that is debatable. Should movies only be reviewed by people who expect to like the movie? In either case, she was not at fault for disliking a movie she wouldn’t choose to watch in her personal life. I’m sure each critic has their own experiences like this and I only single this one case out as it has stuck with me. The anger and cries for retribution were way out of whack with someone not liking a movie.

By its very nature, film criticism is subjective because (a) art can’t have a single objective verdict; and (b) that’s the point – you want to learn someone’s perspective. To me, this seems self-evident. And yet, so many of the public comments added to film reviews and on Twitter are based on the premise that film reviewers could be reaching a single universal objective truth if they weren’t so damned lazy / biased / female.