Everybody else on the internet seems to be talking about review scores, what with the furor over some outlets giving Killzone 2 a less-than-perfect score. I see no reason for us to be different.
So, this week's Destructoid Discusses is on the value of review scores. Are they important? What importance do they have in our buying decisions? How do you respond when a game gets a review other than what you might have assigned or expected?
We're joined this week by tsunamikitsune and we managed to wrangle a pretty good conversation, if I say so myself. Hit the jump and you'll see what I mean. And make sure to contribute your thoughts in the comments.
Conrad Zimmerman: Let's try not to beat a dead horse on how the number should not be the most important factor and that people should actually read the text of a review rather than skip to the number at the end. I think it's safe to say this will never happen.
Jim Sterling: I think the number depends on where it's come from. An IGN 7 is not the same as a Destructoid 7. This is the reason why our reviews tend to be lower on the Metacritic scale, because we invest a lot more value into each of our numbers. For us, a 7 is good, and well worth a purchase for a significant number of people. I only get that impression from Destructoid reviews and Edge reviews, however. Other places seem to have completely devalued anything below an 8.
Dale North: Even more specifically, it depends on which reviewer reviews that game. The flat number from XXX outlet means nothing to me without knowing who wrote it. But I know exactly what a Colette Bennett "8" means, or what a Jim Sterling "5" means.
Jim Sterling: I think knowing the personality of the review is helpful with reviews, and one of the benefits we have of being a blog of individuals as opposed to a collective voice. As you say, you know what a review from Colette or me means. People with the same tastes as our writers can use that to better help them decide if a game is worth looking into. Same goes for people you tend to disagree with.
Jordan Devore: I think knowing the personality of the review is helpful with reviews, and one of the benefits we have of being a blog of individuals as opposed to a collective voice. As you say, you know what a review from Colette or me means. People with the same tastes as our writers can use that to better help them decide if a game is worth looking into. Same goes for people you tend to disagree with.
Brad Nicholson: I'm a cautious buyer and I always will be. Review scores don't mean a lot to me, but that's because I actually read the reviews. The difference between a seven and an eight can be vast from Web site to Web site. Some sites aggregate scores, others assign them flimsily, others (like us) assign meaning behind the number.
Colette Bennett: I utterly ignore review scores. It isn't that I don't care what a reviewer has to say so much as I know that no matter what the general consensus is, it's likely I will not wholly agree with it. Every once in a while if a game I am interested in gets overwhelming low scores across the board, I'll take it into account, but it may not prevent me from buying the game. I've loved a lot of games that have been critical "flops" (Earth Defense Force FTW!).
Jim Sterling: Welcome to seemingly 3% of the gaming population.
Colette Bennett: Happy to be here, Sir. :)
Jonathan Holmes: Yeah, I love reading reviews for the info, but the scores (and the opinion or the reviewers) are usually pretty meaningless to me. I've read a lot of 10/10 reviews that actually helped me to know that, even though the game is technically well crafted, that it's just not for me.
I think the IGN review of Gears (a 10/10 review) actually said something like "The plot is stupid and the gameplay gets really mindless and repetitive, but the kick ass explosions and non-stop brown-ness make this game a must have!"
That was very helpful information.
tsunamikitsune: For the longest time I would use IGN for my reviews, but as I got to playing some of the supposedly crappy games (See: God Hand), I realized that I can't always skip straight to the number score in order to decide my purchases. I continue to use them to get a rough idea of what a game is like when it comes out (assuming Destructoid hasn't covered it already). I don't put so much trust in the scores anymore, especially since I realized that many different people write the reviews for IGN, making it difficult for scores to be compared with one another. I agree with Jim that knowing the personality is important, as I feel much more educated when I read a review by someone who has had experience with the type of game they're reviewing.
Niero: I'd like to chime in on the business of review scores. Even though it's obvious that the general public doesn't give a shit as much as some core gamers might, the publishers definitely do. Some even have payout structures based on how Metacritic scores.
Jim Sterling: And Niero illustrates, as he so often does, why I am proud to serve as his reviews editor.
Jonathan Holmes: For me, adding the score to a review is by far the worst part of the process of doing a review. I'd love to be able to skip it.
I'd also love for people to be able to read my mind without me having to type a single word, but that's not possible. If you want to communicate what you think, you need to use the language, and that's what review scores are; the language of reviews.
tsunamikitsune: I can't imagine how awful it is for you guys to try and assign a number score to your reviews. It seems like it takes a lot of thought that most sites spend little time on (which is evident when you read a review that hates on every aspect of the game, then see that the score is much higher than the review makes it out to be).
Jonathan Holmes: That's being optimistic, Kit.
Honestly, I think a lot of sites (and moreso magazines) assign their review scores based on what people want to hear. Gaming magazines and websites feed off of the excitement of their readers. Financially, they couldn't exist without that excitement. More so, magazines and websites need to make their readers happy in order to stay in business.
Sadly, giving a game 10/10 review scores make people happy. So do 3/10 scores for games that people want to hate.
Jim Sterling: I've tried to make it so assigning a score is as painless as possible. All you have to do is line up the tone of your review with the summary of each number. Of course, that's not an exact and simple science, but I've found it's taken a lot of thought out of the number process. One shouldn't be wasting time agonizing over slapping an arbitrary numerical figure to a review.
Jonathan Ross: Scores are a tough thing. I personally hate giving them, but at the same time I like reading them on other sites. I don't use them as the sole deciding factor in buying shit, but a lot of times they're helpful if I'm deciding between two games. At the same time though, I know that a lot of them are arbitrary and that everyone uses a different scale anyway, but that doesn't seem to stop me from taking them into account. I do overall tend to find the review text overall much more helpful, but I HATE when certain sites just copy/paste the entire review text and score for the PC/360/PS3 versions of the game.
Jonathan Holmes: Oh, it's not your fault Jim, or the review system's. My issue with placing a score on a review is totally my own.
At times in my life when I had to put a number or letter grade on someone else's writing or artwork, I always had a tough time with the process. There is so much subjectivity involved, so many factors to keep in mind.
Sometimes it's easier than others. I had no problem giving Mega Man 9 a 9/10, but some of those Strong Bad games, which were all about two very subjective things (comedy and puzzles), were really hard to assign a score to.
Tom Fronczak: I think a big piece of the rage behind review scores, even if it's subconsciously, is how far the industry has come financially. When you walk into a store, you know every game on the shelf spent millions bringing their idea from concept to completion. I can see a lot of people walking in and saying to themselves "Fuck yeah! This company just spent ten million making this game over two years and it's going to be amazing. I'm going to be blown away." In some ways we want every game we buy to be a 10/10. So it's not so much that a 8/10 is bad, it's the idea that, if the game company spent that much money making it, then why isn't it polished and perfect? If it's not, then why do we work so hard at our daily jobs when there's much less money involved? It sucks that every game on that shelf is $60 bucks, but only maybe only two of them will be a good game -- so why do you have to pay the same for all of them? Even though it's a ridiculous and unrealistic frame of mind, I'm pretty sure a lot of people look at a 6/10 game as 40% of its budget was mishandled. There's a grain of truth to that, because consumers should always demand greatness no matter what, but pieces of art tend to defy normal logic.
Dyson: I just look at the numbers of the review. If the number is higher or lower than what I would have given the game, then I'll read the review. Or if I'm totally bored. I know whether or not I'm going to get a game long before So-and-so Jones from Nowhereville post his finely crafted 1100 word opus. Making me read that junk just find out the the person gave it the same score I would have is a waste of my time. There are no less than a billion people out there chomping on the bit trying to show the world what their idea of what a good review for a game is, and they all miss their mark horribly.
Dale North: First off, OMG tusnamikitsune -- I was out to lunch earlier, thinking about this topic -- thinking about the countless examples of how review scores did not reflect the "score" for me. God Hand came to mind. So high five on that.
tsunamikitsune: High five indeed, Dale. God Hand is pretty high up there on my list of favorite games, so I'm baffled by IGN's review. How can any game that lets you kick a guy in the nuts get such a low score?
Joseph Leray: Once again, I think Tron and I are on the same page here (for the most part). Not that I do a lot of reviewing around here, but I *hate* assigning a number to something. I just wrapped up a review for a game that, while technically pretty unremarkable, I had a lot of fun playing. The games mechanics are really solid, but it's executed rather poorly. Most sites have roundly panned this game (and I can see why), but I found certain parts of it incredibly fun. Trying to find a score that accurately reflects, on the one hand, the fun I had with it and, on the other, it's technical and mechanical short comings was kind of tricky.
Jonathan Holmes: Oh, I agree with you 100% Joseph, if it weren't for the fact that I just can't trust that most reviewers have taste anything like my own.
I've probably over used this example at this point, but after Game Informer gave No More Heroes a 6.5 for being too "sexual and violent", despite the fact that they'd later make GTA IV their game of the year, I pretty much gave up on taking anything away from a reviewer's opinion of a game.
Chad Concelmo: I give this Dtoid Discussion a 9.3 out of 10! <3
Jonathan Holmes: YOU ARE BIAS MAN
Dale North: Replay value is shit.