Note: iOS 9 + Facebook users w/ trouble scrolling: #super sorry# we hope to fix it asap. In the meantime Chrome Mobile is a reach around
hot  /  reviews  /  videos  /  cblogs  /  qposts

Destructoid Discusses: The value of review scores

4:00 PM on 02.11.2009 // Conrad Zimmerman

Everybody else on the internet seems to be talking about review scores, what with the furor over some outlets giving Killzone 2 a less-than-perfect score. I see no reason for us to be different.

So, this week's Destructoid Discusses is on the value of review scores. Are they important? What importance do they have in our buying decisions? How do you respond when a game gets a review other than what you might have assigned or expected? 

We're joined this week by tsunamikitsune and we managed to wrangle a pretty good conversation, if I say so myself. Hit the jump and you'll see what I mean. And make sure to contribute your thoughts in the comments.

Conrad Zimmerman: Let's try not to beat a dead horse on how the number should not be the most important factor and that people should actually read the text of a review rather than skip to the number at the end. I think it's safe to say this will never happen.

I'm more interested to know what effect it actually has on you as a consumer to see that a game received a seven vs. an eight? How much does anticipation of a title color your judgment on whether a game is scored fairly?

Jim Sterling: I think the number depends on where it's come from. An IGN 7 is not the same as a Destructoid 7. This is the reason why our reviews tend to be lower on the Metacritic scale, because we invest a lot more value into each of our numbers. For us, a 7 is good, and well worth a purchase for a significant number of people. I only get that impression from Destructoid reviews and Edge reviews, however. Other places seem to have completely devalued anything below an 8.

Dale North: Even more specifically, it depends on which reviewer reviews that game. The flat number from XXX outlet means nothing to me without knowing who wrote it. But I know exactly what a Colette Bennett "8" means, or what a Jim Sterling "5" means.

As a consumer, review scores have no affect on me as solitary ratings. Together, amassed on a site like Metacritic, they give me a general idea. I love to pull up the site on my phone while I'm out shopping for games. But then again, a overall 50% isn't going to stop me from buying a game either.

Jim Sterling: I think knowing the personality of the review is helpful with reviews, and one of the benefits we have of being a blog of individuals as opposed to a collective voice. As you say, you know what a review from Colette or me means. People with the same tastes as our writers can use that to better help them decide if a game is worth looking into. Same goes for people you tend to disagree with.

Jordan Devore: I think knowing the personality of the review is helpful with reviews, and one of the benefits we have of being a blog of individuals as opposed to a collective voice. As you say, you know what a review from Colette or me means. People with the same tastes as our writers can use that to better help them decide if a game is worth looking into. Same goes for people you tend to disagree with.

Brad Nicholson: I'm a cautious buyer and I always will be. Review scores don't mean a lot to me, but that's because I actually read the reviews. The difference between a seven and an eight can be vast from Web site to Web site. Some sites aggregate scores, others assign them flimsily, others (like us) assign meaning behind the number.

As a consumer, I think a seven means that the game has a glaring flaw that I should consider before purchase. The flaw doesn't have to be detrimental to the game (and it probably isn't considering the game got a seven) it's just there and noticeable. Generally, whenever I assign a seven it's because the game is too short, the story poor, or a specific mechanic doesn't work as much as I would like it to. With an eight, I think you're dealing with a game that has a series of negligible flaws that keep it from reaching a certain level of perfection.

It's all subjective and I need to take that into account as a reader. I need to familiarize myself with the reviewer's other work. Review scores don't help the decision making process -- it's the writing. And that's something I've been pounding into heads since I started writing reviews.

Colette Bennett: I utterly ignore review scores. It isn't that I don't care what a reviewer has to say so much as I know that no matter what the general consensus is, it's likely I will not wholly agree with it. Every once in a while if a game I am interested in gets overwhelming low scores across the board, I'll take it into account, but it may not prevent me from buying the game. I've loved a lot of games that have been critical "flops" (Earth Defense Force FTW!).

That being said, if I do happen to read a review I tend not to react, even if the reviewer is harsh about something I liked a lot. I am able to separate a criticism about something I liked from a criticism directly aimed at me.

Jim Sterling: Welcome to seemingly 3% of the gaming population.

Colette Bennett: Happy to be here, Sir. :)

Jonathan Holmes: Yeah, I love reading reviews for the info, but the scores (and the opinion or the reviewers) are usually pretty meaningless to me. I've read a lot of 10/10 reviews that actually helped me to know that, even though the game is technically well crafted, that it's just not for me.

I think the IGN review of Gears (a 10/10 review) actually said something like "The plot is stupid and the gameplay gets really mindless and repetitive, but the kick ass explosions and non-stop brown-ness make this game a must have!"

That was very helpful information.

tsunamikitsune: For the longest time I would use IGN for my reviews, but as I got to playing some of the supposedly crappy games (See: God Hand), I realized that I can't always skip straight to the number score in order to decide my purchases. I continue to use them to get a rough idea of what a game is like when it comes out (assuming Destructoid hasn't covered it already). I don't put so much trust in the scores anymore, especially since I realized that many different people write the reviews for IGN, making it difficult for scores to be compared with one another. I agree with Jim that knowing the personality is important, as I feel much more educated when I read a review by someone who has had experience with the type of game they're reviewing.

I usually read reviews simply to gather even more information on games I'm already hyped about, so the scores do little to change my opinion in most cases. Honestly, I think scores are pretty worthless aside from giving people something to argue about. Why read a wall of text when you can just get a bolded numeric summary at the end? Maybe reviewers should make the readers work for it by hiding the number somewhere in the bulk of the review. That'll teach them the value of reading.

I must admit, however, that I get a sick pleasure out of skimming down to the score for a crap Wii game when I see them reviewed on Dtoid. A quick laugh at a horrid game always gets my day off to a good start.

Niero: I'd like to chime in on the business of review scores.  Even though it's obvious that the general public doesn't give a shit as much as some core gamers might, the publishers definitely do.  Some even have payout structures based on how Metacritic scores.

Destructoid has had ad campaigns pulled because of our review scores.  Nobody's ever said to me, hey, pull that number or we'll pull the ads because we work though ad networks that do that stuff for us.  When I see a hard pause on a campaign the day after a review when up and ask what happened, they'll say the client had a change of heart. And you know what?  They have the right.  They don't want to look like fools putting ads on a site that hated their game.  It's a waste of their money.  I'm not going to cry on their lap or demand that they appreciate our editorial integrity  -- we don't want to be a high maintenance bitch, either.  They don't hire media buyers who's job requirements call for that. 

A respected person in the game industry once said to me "Why do you guys do review scores?  You're just hurting your advertising revenue and not getting any traffic for it."  Another guy who used to work for us would often beat me up about our 10 point system -- why publish numbers at all?  My answer to both questions was simple:  "Because enough of the readers appreciate it."  When Destructoid committed to publishing reviews we committed to becoming part of that ecosystem. 

So, this is a burden the editors bear -- something we all believe strongly enough to lose money over and put us in less than favorable relationships with some of the publishers we love the most.  And that's the reality of that number; the weight of that number is worth real money and real problems.  If you read anyone in the industry who doesn't publish a review number, this is the real reason.  They're being pussies -- and better businessmen than me.  And that's fine.  That's why we're Destructoid and they're not.

Jim Sterling: And Niero illustrates, as he so often does, why I am proud to serve as his reviews editor.

Jonathan Holmes: For me, adding the score to a review is by far the worst part of the process of doing a review. I'd love to be able to skip it.

I'd also love for people to be able to read my mind without me having to type a single word, but that's not possible. If you want to communicate what you think, you need to use the language, and that's what review scores are; the language of reviews.

tsunamikitsune: I can't imagine how awful it is for you guys to try and assign a number score to your reviews. It seems like it takes a lot of thought that most sites spend little time on (which is evident when you read a review that hates on every aspect of the game, then see that the score is much higher than the review makes it out to be).

I think most sites decide their review scores by wearing a blindfold and throwing a dart at a dartboard. It may not be as accurate, but it's probably a lot more fun.

Jonathan Holmes: That's being optimistic, Kit.

Honestly, I think a lot of sites (and moreso magazines) assign their review scores based on what people want to hear.  Gaming magazines and websites feed off of the excitement of their readers. Financially, they couldn't exist without that excitement. More so, magazines and websites need to make their readers happy in order to stay in business.

Sadly, giving a game 10/10 review scores make people happy. So do 3/10 scores for games that people want to hate.

Jim Sterling: I've tried to make it so assigning a score is as painless as possible. All you have to do is line up the tone of your review with the summary of each number. Of course, that's not an exact and simple science, but I've found it's taken a lot of thought out of the number process. One shouldn't be wasting time agonizing over slapping an arbitrary numerical figure to a review.

I personally feel scores do have an important place in the overall review process, but if you spend too long deliberating on it, you're doing something wrong. Your text should tell you what the score is.

Jonathan Ross: Scores are a tough thing.  I personally hate giving them, but at the same time I like reading them on other sites.  I don't use them as the sole deciding factor in buying shit, but a lot of times they're helpful if I'm deciding between two games.  At the same time though, I know that a lot of them are arbitrary and that everyone uses a different scale anyway, but that doesn't seem to stop me from taking them into account. I do overall tend to find the review text overall much more helpful, but I HATE when certain sites just copy/paste the entire review text and score for the PC/360/PS3 versions of the game.

Jonathan Holmes: Oh, it's not your fault Jim, or the review system's. My issue with placing a score on a review is totally my own.

At times in my life when I had to put a number or letter grade on someone else's writing or artwork, I always had a tough time with the process. There is so much subjectivity involved, so many factors to keep in mind.

Sometimes it's easier than others. I had no problem giving Mega Man 9 a 9/10, but some of those Strong Bad games, which were all about two very subjective things (comedy and puzzles), were really hard to assign a score to.

Tom Fronczak: I think a big piece of the rage behind review scores, even if it's subconsciously, is how far the industry has come financially. When you walk into a store, you know every game on the shelf spent millions bringing their idea from concept to completion. I can see a lot of people walking in and saying to themselves "Fuck yeah! This company just spent ten million making this game over two years and it's going to be amazing. I'm going to be blown away." In some ways we want every game we buy to be a 10/10. So it's not so much that a 8/10 is bad, it's the idea that, if the game company spent that much money making it, then why isn't it polished and perfect? If it's not, then why do we work so hard at our daily jobs when there's much less money involved? It sucks that every game on that shelf is $60 bucks, but only maybe only two of them will be a good game -- so why do you have to pay the same for all of them? Even though it's a ridiculous and unrealistic frame of mind, I'm pretty sure a lot of people look at a 6/10 game as 40% of its budget was mishandled. There's a grain of truth to that, because consumers should always demand greatness no matter what, but pieces of art tend to defy normal logic.

Rev wrote a great article a while ago about the 10/10 game review, and how no game or movie or other piece of art is ever a 10, that it's a mythic unattainable beast with no flaws. It had a lot of great points in it that I agree with, but no matter how much support those arguments have, I have to admit I still really like numbered reviews. I think Jim and Dale hit it on the head -- one person's 9 is not another reviewer's 9. You need to trudge through a lot of crappy or okay reviews and reviewers and find a few people with the same mindset as you. Someone who is a similar type of gamer, someone who looks for the same things as you in games, and someone with the same set of expectations and demands from the gaming industry.

That's why I love our extra review staple. At the end of the day, most gamers just want to know if they should rent the game, buy it, completely avoid it, or if they should expect the game to win awards. It's a good way to cut out all artistic interpretation and put your foot down on exactly how you feel about the game you just played.

Dyson: I just look at the numbers of the review. If the number is higher or lower than what I would have given the game, then I'll read the review. Or if I'm totally bored. I know whether or not I'm going to get a game long before So-and-so Jones from Nowhereville post his finely crafted 1100 word opus. Making me read that junk just find out the the person gave it the same score I would have is a waste of my time. There are no less than a billion people out there chomping on the bit trying to show the world what their idea of what a good review for a game is, and they all miss their mark horribly.

Dale North: First off, OMG tusnamikitsune -- I was out to lunch earlier, thinking about this topic -- thinking about the countless examples of how review scores did not reflect the "score" for me. God Hand came to mind. So high five on that.

And, back to the discussion, as far as Destructoid review scores goes, Jim has made it really easy to drop a number on a review. As a member of Dtoid's review crew, I used to hate numbering these things. Now it's no big deal at all. Each number actually means something here. I can go back and look at a number that any of us have given for a game and feel really good that that number means something... to us, at least.

tsunamikitsune: High five indeed, Dale. God Hand is pretty high up there on my list of favorite games, so I'm baffled by IGN's review. How can any game that lets you kick a guy in the nuts get such a low score?

The Buy it/Rent it/Avoid it scale is amazing. That combined with the knowledge of the reviewer's taste usually gives me a pretty good idea about how much I might like the game in question.

Now that I think about it, scores do have a bit more value than I originally gave them credit for. When I scroll down to the bottom of a review and notice a high score for some game I've never heard of or previously didn't care about, I usually find my way back to the top of the page, curious about what exactly made this game so great. On these occasions, I actually read the review instead of just glancing at the score. This worked for Lock's Quest and CarnyVale Showtime, both of which I hadn't heard of until the Destructoid reviews. They're both great games that I never would have noticed if not for the high scores they were given.

Speaking of CarnyVale Showtime, playable demos are great after my interest has been piqued by a good review (or even a good score). The trial certainly certainly helped when I was on the fence about Fireplace.

Joseph Leray: Once again, I think Tron and I are on the same page here (for the most part). Not that I do a lot of reviewing around here, but I *hate* assigning a number to something. I just wrapped up a review for a game that, while technically pretty unremarkable, I had a lot of fun playing. The games mechanics are really solid, but it's executed rather poorly. Most sites have roundly panned this game (and I can see why), but I found certain parts of it incredibly fun. Trying to find a score that accurately reflects, on the one hand, the fun I had with it and, on the other, it's technical and mechanical short comings was kind of tricky.

Where I disagree with Tron, though, is when it comes to what reviews are good for. Yes, I read reviews for information, but also to see how the game makes people feel. I don't want to read about what the controls do, or the minutiae of an RPG's battle system -- I want to know if it makes me feel good, if it's fun. I don't care if the textures are all bland -- I want to know if it adds to or detracts from the experience of the game.

Jonathan Holmes: Oh, I agree with you 100% Joseph, if it weren't for the fact that I just can't trust that most reviewers have taste anything like my own.

I've probably over used this example at this point, but after Game Informer gave No More Heroes a 6.5 for being too "sexual and violent", despite the fact that they'd later make GTA IV their game of the year, I pretty much gave up on taking anything away from a reviewer's opinion of a game.

Chad Concelmo: I give this Dtoid Discussion a 9.3 out of 10! <3

Jonathan Holmes: YOU ARE BIAS MAN

Niero: Fanboy.

Dale North: Replay value is shit.

Conrad Zimmerman, Moustache
 Follow Blog + disclosure ConradZimmerman Tips
An avid player of tabletop and video games throughout his life, Conrad has a passion for unique design mechanics and is a nut for gaming history. He can be heard on the comedy podcast () and str... more   |   staff directory

 Setup email comments

Unsavory comments? Please report harassment, spam, and hate speech to our community fisters, and flag the user (we will ban users dishing bad karma). Can't see comments? Apps like Avast or browser extensions can cause it. You can fix it by adding * to your whitelists.

Status updates from C-bloggers

The Dyslexic Laywer avatarThe Dyslexic Laywer
Is it too late to write about the waifu wars? I don't really have one but there is a female character I have in mind that I want to talk about.
StriderHoang avatarStriderHoang
Bayo has high execution barrier and largely unsafe moveset but Witch Time is a huge saving grace. I bet people are double bitter with Corrin being in the game and being pretty reliable in terms of skillset. Definitely a safer and stronger choice than Bayo
Jed Whitaker avatarJed Whitaker
Streaming some Unravel then perhaps some Firewatch. Don't not come. [url][/url]
Agent9 avatarAgent9
Just got my Wind up Ifrit minion. I couldn't sell it, it was too adorable.
Agent9 avatarAgent9
Just got my Wind up Ifrit minion. I couldn't sell it, it was too adorable.
Parismio avatarParismio
I was playing Third Strike on PS3 with my PS4 controller and I tried using the dpad for the first and noticed that it doesnt take corner directional inputs. Is this normal for ps4 controllers on ps3?
Larxinostic avatarLarxinostic
I swear, it makes sense in context..... Kinda. Hmmm. Okay, not so much. [img][/img]
Agent9 avatarAgent9
Almost done with my Waifu wars blog. pretty happy with how it turned out.
SeymourDuncan17 avatarSeymourDuncan17
Time to scream and shout. It's Nanako cosplaying as her big bro! <3
Mike Wallace avatarMike Wallace
Bernie Sanders vs. Donald Trump is like Gandalf the White vs. Handsome Jack.
Sir Shenanigans avatarSir Shenanigans
Skellige is so cool! It's like the land of Valhalla Rising.
Shinta avatarShinta
God damn, Bernie Sanders is just killing it with this speech. Hitting basically every point. He even used the word "oligarchy." Probably the first time I've ever heard that word uttered on CNN. I think a lot of people in power are shitting their pants
Pixie The Fairy avatarPixie The Fairy
In my haste to finally factory reset my tablet, I erased a blog I had worked on. Thankfully, it's fresh in my mind. It's another MGS blog, but it goes the opposite way of my last MGS blog. Pray this guy is not your husbando, for he is shit.
Sir Shenanigans avatarSir Shenanigans
Just ate a disgusting amount of sugary wonders in a Fat Tuesday blowout. Chocolate (birthday) cake, Oreos, brownies, cookie dough, and some creme brule thing. Satiation by way of eat-'til-you-puke is what Shenanigans says!
LaTerry avatarLaTerry
Is there any real difference between the PS3 and the PS4 versions of Valkyria Chronicles?
Shinta avatarShinta
KnickKnackMyWack avatarKnickKnackMyWack
Say whaaaaaat?
Gundy avatarGundy
Voting for Broforce made me think of the most American person that could ever exist. President Michael Wilson!
Fuzunga avatarFuzunga
By the way, that IGPX collection is a new release. It's the first time the show is available in a complete package, and the first time it's been available in any format in about 10 years. [url][/url]
more quickposts



Invert site colors

  Dark Theme
  Light Theme

Destructoid means family.
Living the dream, since 2006

Pssst. konami code + enter

modernmethod logo

Back to Top

We follow moms on   Facebook  and   Twitter
  Light Theme      Dark Theme
Pssst. Konami Code + Enter!
You may remix stuff our site under creative commons w/@
- Destructoid means family. Living the dream, since 2006 -