June 28, 2014

Times of India's Movie Reviews: A Scam of Freakonomical proportions



I have been a loyal TOI reader since I was about ten. Since then, my daily routine has revolved around reading the newspaper. I was also, in my school days, a TOI Young Journalist for two years and fleetingly considered a career in journalism. 


If you read a paper for a decade and a half as I have (and for several decades as have my parents) you grow addicted to it… you begin to take it seriously. The newspaper business is all about loyalty. People will tell you their breakfast/ tea didn’t go down that well if for whatever reason, they could not grab hold of their favourite newspaper (which was, by the way, a common occurrence during all my childhood vacations with my mother.)


Anyhow, back to me. What I did, when I did not become a journalist, was to become an engineer and an MBA. What that gave me in turn, was an introduction to several simple yet powerful mathematical tools… some of which I will showcase here today…


I have been, for some time, particularly pained by the disparity in TOI’s movie ratings versus ratings by independent reviewers like Masand/ Anupama Chopra. From a point where you could blindly trust TOI to rate a movie honestly, things have come to such a head that you need to verify the TOI’s movie ratings with more independent sources before making a movie-watching decision.


So here’s what I did using the modicum of mathematics I remembered from my engineering and MBA days. I painstakingly took down the ratings of 100 Hindi movies from TOI and from IMDB in an excel and then, subtracted the IMDB Rating from TOI’s Rating to create a dependent variable I shall call “Gap in Ratings”. I also googled extensively for the movies’ production budgets (this removed three movies from my data set whose budgets I could not find on Google: Filmistaan, Rangrezz and O Teri). On top of this, I added an (econometric) layer of whether the movie had a “bankable” star like one of the Khans (Saif included), Amitabh Bacchan, Akshay, Ajay Devgan, etc... And then, I ran a linear correlation and several regressions (going up to the power of seven), just to see if there was a pattern with which the “Gap in Ratings” between TOI and IMDB varied.


Just a snapshot of what the data looked like is here:



I found that there was a 47% correlation between the gap in movie ratings on TOI vs IMDB and the budget of a movie. Now what that technically means is that 47% of the value of the variable “Gap in movie ratings on TOI vs IMDB” can be explained by the independent variable “Budget of a movie” while the remaining 53% is determined by other factors. But 47% isn’t really strong enough; there’s still 53% that’s explained by factors we do not statistically understand (or haven’t accounted for).


What, however, is infinitely more interesting is that if you take the average gap in ratings between TOI and IMDB for movies with non bankable stars, it comes to +0.38 points on a scale of 10. However, throw bankable stars into the mix and the average gap in ratings between TOI and IMDB goes up to +1.11 on a scale of 10


That’s a 190% jump in the average gap in ratings between TOI and IMDB if a movie has a bankable star. Meaning, in non mathematical terms, the disparity in ratings given by TOI vs IMDB on average goes up by 190% if the movie has a bankable star. 


(For the record, I have counted the Khans including Saif, Sanjay Dutt, Akshay Kumar, Ajay Devgun, Hrithik, Ranbir, the Deols and Emraan Hashmi as bankable stars… bankable stars, by definition, are the marquee stars whose films always make money or are supposed to always make money, anyway)


Since averages can sometimes be skewed by a couple of data points, let us now look at medians. So if you take the median gap in ratings between TOI and IMDB for movies with non bankable stars, it comes to +0.2 points on a scale of 10. However, throw bankable stars into the mix and the median gap in ratings between TOI and IMDB goes up to +1.3 on a scale of 10. 


That’s a whopping 550% jump in the median gap in ratings between TOI and IMDB, if a movie has a bankable star. Meaning, in non mathematical terms, the disparity in ratings given by TOI vs IMDB at the median goes up by 550% if the movie has a bankable star.


Another interesting fact – if you look at movies with bankable stars, the minimum rating TOI has given (for the 97 movie data set going back a couple of years) is 2 (out of 5) whereas the least rating for a movie with a bankable star on IMDB is 0.85 (out of 5).


That’s a 135% higher minimum review, seemingly guaranteed, by the TOI.


Since numbers cannot lie, it is highly likely that the rumours doing the rounds in media circles are true and TOI actually charges money for maintaining a minimum guaranteed review. Consider this – Humshakals, a movie panned universally by almost all critics, got three stars from TOI. Himmatwala got 2.5, as did ‘The XPose’ and Rangrezz – all movies which were universally buried by the rest of the movie critic universe. The list goes on and if you are really interested in doing your own data analysis or going through my numbers, reach out to me at vaibhavrainmaker@gmail.com.


So where does this take us?


One – TOI is most likely on the take whenever there is a big budget movie with a big star involved to give them a Minimum Review of at least 2.5 or 3. If the movie is terribly terribly terribly terrible, the Minimum Review guarantee falls to 2 (eg: Chatur Singh).


Two – if the movie has a bankable star, just shave off 0.55 points off TOI’s review from a statistical point of view to arrive at its actual watchability potential.


And, dear Times of India, if you are reading this, you just lost a loyal reader. 

PS: Data analysis done on 97 hindi movies with available budgets. 41 of these had “bankable” stars.
 

2 comments:

  1. They can't help but randomize based on some variables when the case is like that of Bollywood (where there are hardly any movie (out of 100) worth even 2 stars).

    ReplyDelete