I was reading Riobux’s blog on review scores and thought I’d give my thoughts on the whole debacle. Before I start, however, I would like to point out that I’m not a games journalist, nor do have any interest in becoming one; these comments are from a hobbyist, not a professional, so you can make of them what you will. First of all, I recognise that review scores are necessary. Reviews would ultimately be useless if there was no quantifiable way of comparing them with each other, my problem lies with the undue importance most sites give them and also the act of giving half stars and decimal places. I’ll deal with the latter first:
Given how much they used to bang on about it, you’d have thought that the difference between accuracy and precision was the most important thing on the A-level Chemistry syllabus. 3 or 4 years on it’s the only thing I remember about the subject and it is relevant to my point: If something is accurate, it is repeatable. If we wanted to find whether or not a review accurately reflected general opinion of a game we would compare it to the mean of all other reviews of the game and if it was the same we could agree that the review was indeed accurate. It should be obvious then, that an accurate review is neither desirable nor strictly possible. If a game has 3 reviews, all drastically different, it is far more useful to the consumer than 3 reviews all singing the game’s praises; yet when a site gives a review which bucks the general trend they often receive a negative response. Obviously fan-boyism has a large part to play in this, but I think a misguided attempt to produce precise review scores from many sites also has a very large part to play. Just to clarify, precision is nothing more than the use of a smaller scale, i.e. a score of 1% or 1.0/10 is more precise than a 1/5 or 1/10. The problem in using these more precise scales is that they imply a level objectivity and accuracy, when really, especially when you get down to decimal places, these numbers have ceased to reflect anything valuable about the experience. I’m just going to bring up IGN’s ORAS review as an example: 7.8/10. What does that even mean? The game was 0.2 enjoyments per second short of an 8? Similarly how can the writers of the former ONM justify giving Super Mario Galaxy 2 98/100? I recently had to write a Skeleton report on 4 individuals, all had been highly truncated by recent building work and when it came to assigning ages I assigned 3 as fully mature and placed a fourth in a 10 year age bracket. I also explained why I had made these decisions and a big part of this was acknowledging how precise the evidence allowed me to be. Much like these age brackets, review scores are discrete categories based on subjective observations, there will be variation within them and that is not a problem as long as the reader knows why you placed them in these categories. If ORAS didn’t deserve an 8 it should have been given a 7, SMG2 a 9, these are much more useful reflections of the games overall quality.
Next onto the precedent review scores are given. Dtoid recently gave #IDARB 8.0/10, seeing as it was free me and a few friends decided to give it a go. We won our first game and it was close, with a nail biting final quarter we left in the knowledge that at the end of the day the better team had won. We then all unanimously agreed that the game was utter shite and went back to playing Diablo 3. That 8.0/10, the first thing I see about #IDARB when I look on Destructoid’s homepage was downright unhelpful, and to a certain extent misleading. I wouldn’t give the game 8.0/10, I wouldn’t even give it 7.9/10. I’d give it 2/5, it was alright, I enjoyed myself to a certain extent but it was too chaotic and I have no desire to play it again. However, thanks to Destructoid’s practice of plastering a gurt big picture of the person who reviewed it at the top of the review I now know that Jason Faulkner and myself have very different tastes in videogames. As I read more of his reviews I will agree and disagree with the points that he makes I’ll construct a lens through which to view the rest of his reviews, giving his opinions context and making them far more useful than that initial #IDARB review. Because Destructoid has allowed me to identify individual reviewers and features them so prominently I can make more nuanced decisions based on their reviews, Videogamer.com, my other go to site for reviews, is the same.
Summarising my views on reviews then:
The author should be credited even on the link to the article; this information is ultimately more valuable than the score, and there should be a concerted effort from the company to allow each reviewer an individual presence within the overall company. Reviews should be given on a scale of 0 to 5, and there should be no decimal places. Using the categories described below the majority of games will fall into the 3/5 bracket, however that is not an issue: ultimately differentiating between the games in the category will largely come down to personal preference. An RPG fan will know that a 3/5 RPG will be enjoyable but that a 3/5 FPS can probably be given a miss and vice versa.
0/5- Bad to god awful: there is really no need to make the distinction between the two, you want to play neither.
1/5- Bad with redeeming qualities: The game is bad, but has a particular feature which still makes it stand out.
2/5- Average: Serviceable, nothing to write home about, you were mildly aware of having fun while you played it but will never play it again nor will you remember it in 2 months’ time.
3/5- Good: This will be inherently the largest category and will have the most variation, from games with great ambition but which are held back by poor execution to games with very little ambition but are so well made you will almost certainly have fun. All games in this category however will have had a net positive impact on your day, at the same time however they will do little to stand out from the rest.
4/5- Great: The makings of a fantastic game but with one or two niggling issues which hold it back, meaning it falls just short of being a game GOTY contender.
5/5- Excellent: A game where the flaws are completely insignificant compared to what it does right, the sort of game you finish, and are fully aware you could start again right then and still enjoy it all over again.