BY ALEX BARNARD
Metacritic.com is a useful site for consumers that are looking for an assortment of review sources. The site takes all major reviews for a specific game, movie, album, or TV show, and aggregates their scores together into one number called the Metascore. Largely due to the high $60 cost, consumers typically assign more value to a video game review, and in response video game publishers are fixated on getting the best reviews possible. While high sales figures correlate with positive reviews, the Metacritic score is being over valued by game publishers and fails to give the consumer adequate information.
Over the past few years game publishers have increased their focus on earning a high Metascore. Several companies have included the requirement of earning at least an X/100 in the contracts for their employees. Obsidian Entertainment, developer of Fallout New Vegas, was required to achieve a score of at least 85 on Metacritic to earn a bonus from their publisher, Bethesda Softworks. Despite shipping over 5 million copies within the first month of release, no one was given a bonus because the Metascore was an 84. For a publisher to deny a company a bonus for a game that sells extremely well just because the game was short of the desired Metascore by one point is ludicrous. No other industry gives its content creators such impractical critical reception requirements.
As publishers began to make their interest in Metacritic public, several companies stocks began to see the negative effects due to shareholders worrying about Metascores. When THQ’s Homefront was released, stocks plummeted from $6 to $4.75 a share right after the Metascore was set at 70. With this score falling in Metacritic’s good but not great range, analysts predicted that the game would lose money and THQ’s shareholders panicked preemptively. Overtime Homefront went on to ship 2.6 million copies and analysts then predicted that the game would turn a profit. The initial source of panic caused by the Metascore didn’t seem to make a drastic difference in sales figures, but it hurt THQ’s reputation. Earlier this year, THQ went into bankruptcy and most of its properties were liquidated including Homefront, which was sold for $500,000.
Metacritic’s score system is fundamentally broken. Reviewers do not all use the 1-100 score system, and because of this the Metascore conversion may not always fit the tone of the average review. For a review in which a grade ranking is given Metacritic changes scores to fit their system, so a “B” is 75/100, a “C” is 50/100, and a “D” is 25/100. While some reviewers feel that a 50/100 is an average game, others argue that a 70 is average. Because of this, two reviewers with the same feeling on a game might have their scores differ greatly from one another. Each publication typically explains what each score means specifically to them, but Metacritc fails to take this into account when calculating its Metascore. A review score is most useful as a quick means of comparison from product to product; the actual text of a review holds far more importance. Although the Metascore tries to imply that review scores are an exact science, they give a false sense in the difference of scores when focusing so much on a number.
Metacritc takes the “average” of reviews to form its Metascore. The reason it’s an “average” and not an honest average of scores is that each publication is weighed differently when calculating their score. Better-known publications and well-written reviews are stated as to having more weight when calculating the Metascore, but by keeping specifics secretive the public can never be sure how accurate the score really is of the general quality of a game. On Metacritic’s own list of frequently asked questions they make the bold statement that they will “absolutely not” tell how each of the different critics are weighted in their formula. The site’s lack of transparency is detrimental to its credibility in establishing an average score.
Researchers at Full Sail University sought find out how Metacritc’s averaging works. Their study first found a .72 correlation (a very strong correlation) between the Metascore and the sales figures for a game. While this does nothing to show that a high score on Metacritic causes high sales, it does mean that a Metascore can be indicative of sales. Startling to many, they claim to have found that certain popular publications like IGN were weighed up to five times more than other smaller publications. In a reply to the study Metacritic called their approximations “wildly, wholly inaccurate.” With Metacritic still being shady in how much each publication is valued, they failed to provide any evidence to point out how Full Sail’s study may be inaccurate and instead causes even more skepticism on the site’s practices.
Metacritic has done a lot of damage to the video game industry. Its shady weighted average doesn’t necessary reflects the quality of a game, and should not be used as a scientific measure of quality (although many publishers are unfairly considering it just that). Many have been hurt by an industry obsession with the Metascores, shares have dropped rashly and developers are refused bonuses they arguably deserve for financial success. Each score on Metacritic should be taken with a grain of salt, a game with a 90 is not necessarily better than one with an 80, and it is unfair to journalists for Metacritic to dumb down their reviews to more than a simple number. While the site itself can be a good place to find reviews, its Metascore system needs to be taken less seriously. Especially when how the score is calculated is unethically hidden from the public.
