Skip to content

Review Scores Are For Fanbois and Site Owners

February 20, 2012

In the gaming industry, ranking jobs by desirability, I would say that second from the bottom is the job of game reviewer. It is surpassed for undesirability only by the job of game tester. It is a thankless job that I would wish on no one. The sales pitch is that reviewers get the games early and get paid to play them. I do not exaggerate when I say, there are games that you couldn’t pay me to play. I would rather go to the dentist.

Though criticism of review scores is a cyclical affair*, most recently, Simon Parkin and Jim Sterling were called to task for their respective scoring of Uncharted 3 and Gears of War 3. I was going to wait for the current wave of interest to ebb, but Penny Arcade’s Tycho just highlighted a whole new volley of interviews on the subject. It is time for me to say what only a few seem to understand or are willing to admit.

The unfortunate fact is that Metacritic scores can be cause for individuals being canned and studios being shuttered. They matter to the publishers, regardless of their merit. Yet, of all the ways in which a game can be described, a single number or letter is the least desirable. The number itself is inherently arbitrary, no matter how “consistent” the author is within his/her own scale. Furthermore, though reviewers of all major sites include a written account of their experience with the game, the reality is that many consumers and industry stakeholders often jump straight to the numerical value at the end of the review. Metacritic makes things even easier by averaging all those numbers and letters for them.

In the movie industry, Siskel and Ebert knew what they were doing when they employed their thumbs up or down. It was a simple pass/fail system and they saved details and impressions for their dialogues and diatribes.

I think the video game industry would be served well to copy the thumbs up/down, rotten/fresh, pass/fail approach, as at least four sites do: ArsTechnica, Crispy Gamer, Kotaku, Gamecritics. It allows reviewers to focus more on the anecdotal and experiential and free them from the confines of a rating scale. It helps the consumer and industry stakeholders to focus more on the impressions of the reviewer, rather than encourage a speedy jump straight to the numerical value from a scale which represents different criteria for every reviewer. Dare I say, providing a thumbs up type rating might even serve to bring games closer to that elusive, and coveted “art” designation. (Imagine giving John Williams an B- for his Superman theme because there was no electric guitar, or Monet only an 8 out of 10 for his Water Lilies because he had painted them before. I can hear artists and art critics, internationally and in each their own languages singing in chorus,GTFO.)

This autumn, I stopped reading reviews that had scores attached to them. I no longer give site “hits” to any of those reviews or their authors, controversial or not. Scores serve only two purposes, neither of which is of benefit to the industry or its customers. I no longer hold any pity for any reviewer who complains that they are being unfairly accused of “bias” in their scores. Using any score system is unnecessary, unhelpful, and unconscionable. If Metacritic wants to objectify our wonderful games, let them. Journalists don’t have to assist them by providing a value.

Related Articles:
http://www.industrygamers.com/news/feature-metacritics-industry-impact/
http://www.gamesindustry.biz/articles/marketing-influences-game-revenue-three-times-more-than-high-scores
http://online.wsj.com/public/article/SB119024844874433247-EnpxM1F6fI9YZDofC7VnyPzVrGQ_20070920.html
http://en.wikipedia.org/wiki/Metacritic#Influence_of_game_metascores
http://www.giantbomb.com/news/when-a-mostly-positive-review-becomes-controversial/3764/
http://www.eurogamer.net/articles/2011-10-21-uncharted-3-drakes-deception-review
http://www.eurogamer.net/articles/2011-10-28-naughty-dog-lead-game-designer-discusses-uncharted-3-review
http://www.eurogamer.net/articles/2011-09-14-gears-of-war-3-review
http://penny-arcade.com/2011/11/02
http://www.gamepro.com/article/news/223005/editorial-cliffy-b-calls-eurogamer-haters-for-gears-of-war-3-review-score/
http://www.destructoid.com/cliffy-b-upset-by-hateful-8-10-gears-of-war-3-reviews-211562.phtml
http://www.destructoid.com/dice-battlefield-3-will-be-judged-by-review-scores-214403.phtml
http://www.vg247.com/2011/09/16/finishing-the-fight-cliff-bleszinski-on-gears-3-and-the-future/
http://www.gamefront.com/gears-of-war-3-perfect-review-scores-by-divine-right/
http://nohighscores.com/node/1726
http://www.gamasutra.com/view/feature/6558/five_minutes_of_demons_souls.php

*In an article I read a couple of months ago, someone mentioned the cyclical nature of games review criticism. I cannot find the reference though my personal experience as a consumer of games news supports the claim.

 

UPDATE:

http://arstechnica.com/gaming/news/2012/03/why-linking-developer-bonuses-to-metacritic-scores-should-come-to-an-end.ars

UPDATE 2:
http://penny-arcade.com/report/editorial-article/the-power-of-review-scores-why-critics-have-more-control-than-we-think1

Advertisements
5 Comments
  1. Blankman permalink*
    February 20, 2012 9:52 pm

    Awesome read, Mike! A lot of solid points about the broken format of game reviewing. I find myself relying less and less on Metacritic and game review sites, in general. There are just too many inconsistencies when there are individual’s subjectivity are involved. It’s a shame that developers’ livelihoods are tied to whim of “reviewers.” Many of these so-called reviewers are more interested in hits on their sites by stirring up controversy than actual unbiased opinions. Whether the reason is to remain in favor of publishers for access to future titles or that publishers have taken advertisements on their employer’s game sites, it nevertheless clouds judgement. Just this past week, Activision threatened to blacklist a UK game site, Gamehouse, because they refused to take down an article leaking Black Ops 2. In order to stem any backlash, Activision chalked it up to a simple “misunderstanding.” Let’s not even bring the Jeff Gerstman (GameStop) debacle to further illustrate the employer coercing actual games journalists in changing their reviews to something more favorable to please advertisers.

    Good stuff, Mike!

  2. February 21, 2012 4:48 am

    I give this article a C+, and a 35.7 on my personal 48 point scale.

    Also, the Gamepro link is no more. Takes you to a default Gamepro page on the pcworld site.

    • 3Suns permalink*
      February 21, 2012 12:34 pm

      LOL

      Yes, in fact, even the GameFront article no longer exists, which may well be indicative of the shame the authors and subjects felt after they wrote/spoke. The links shall stay, as tag evidence of the ludicrous, cyclical problems with review scores that serve only to bring in site hits and please or anger fanboys.

  3. February 23, 2012 4:11 am

    I get what you are saying……..But!!!

    Most of the time scores actually work to gauge an at a glance “Should I look into this?” train of thought.

    The scores themselves might seem arbitrary, but at the same time help distinguishing between games more so than a “Pass/Fail” System.

    For example games like “The Darkness 2” are OK, but if I say OK to buy or rent? Or if I just say it’s in the 75-80 range out of a 100. Comparable to average games like Homefront, etc.

    but ah wait you never played Homefront, so then there’s no an easier way to reference than just a score.

    I mean all reviews are subjective, there’s no way for anyone to get a good grasp on a game unless you read several reviews, but without a numeric value things would get harder, instead of easier to know which one to dig deeper for. Hard enough as it is with different sites having different metrics, etc.

    and when several releases lineup it makes a difference having to wade through all the possible choices.

    I don’t argue my way out of ratings when I buy other stuff online, usually a good rating is a good flag for something worthy to buy and research more.

    and games are not Monet or a Movie score. They are built more like a construction project, rather than an artistic muse. I picture it more like a nicely decorated living room, rather than a museum.

    and many times the developers themselves do no even like or play the game. Much less the suits that are behind the enterprise making money out of this. I think there’s very few in the industry who even care about the reviews.

    I mean what better example than Skyrim, the “GOTY”, we basically had to trade it in because it froze all them time after a certain amount of time. No matter how many characters you make. That’s unplayable garbage that wouldn’t get passed up, if they actually played their games.

  4. 3Suns permalink*
    February 23, 2012 2:13 pm

    Frenik, ultimately, it comes down to this opinion.

    I think it is wrong to assist people in making a judgement based on a 1 second glance at a score. However, to warn people away from garbage games by giving them a “Don’t buy” IS a service to customers, yet is not unfair to lazy developers. Second, the scoring is completely subjective, indefensible rubbish. Yet by it, developers’ fates are sometimes decided: not by the comment that the story is thin, or the controls are wonky, but by the 6/10 – regardless of sales or quality of game. And that is utter bullshit. I have too much respect for the developers, and love for the hobby to support this practice any longer.

Comments are closed.

%d bloggers like this: