Marc Doyle, co-founder of Metacritic, was given an opportunity to actively defend his site's methodology and its role in the industry on the latest A Jumps B Shoots Podcast and he thinks he's got the answer to our review score woes.
How do we achieve review score balance? "Review all the sh*t." Doyle states that reviewers need to be more aware of the fact that every score serves a purpose. We all know the differences between an eight game and a nine, but who can really differentiate between a three and a four? Critics need to review everything (not just AAA titles) so that we can, as Doyle puts it, "get that precision on the low end to reflect the precision on the high end."
In order to truly understand how scores should work, we need to review every game and build a better concept of "average."
Head over to A Jumps B Shoots and give the full audio a listen for more of Doyle's insight on game scores, weighted metascores, and how exactly publications are picked for and dropped from Metacritic. It's really a great interview and a welcomed look at their side of these ongoing issues.
Episode 12 – Critically Speaking [A Jumps B Shoots]
| BBcode help |
| [b]Bold text[/b] |
Bold text |
| [i]Italic text[/i] |
Italic text |
| [url]http://www.dtoid.com/[/url] |
http://www.dtoid.com |
| [url=http://www.dtoid.com/]Web link[/url] |
Web link |
|
| [img]http://www.example.com/robot.jpg[/img] |
 |
Post a comment! You can also post a photo below:
Comment with Facebook
Click connect and comment instantly!
|
Comment with Dtoid
New? SIGN UP - it takes 5 seconds
|
33 comments | showing # 1 to 33
|
Comment with Facebook
Click connect and comment instantly!
|
Comment with Dtoid
New? SIGN UP - it takes 5 seconds
|
Comments policy
Destructoid is an open discussion community. You don't need to "audition" to post a comment - just speak your mind. We respect differing opinions on the site, so have at it. Be smart, funny, insightful, clueless, or cute -- but back it up with substance. Keep your cool, keep it fun. We only ask that you act respectfully and above all: don't be a troll and ruin it for everyone else. Don't bring down gamers or we'll, you know, gently shoot you in the face and stuff you into a flaming mailbox. Each comment is your opportuntity to make this community awesomer. Is that even a word?
Avoiding the banhammer only requires common sense: spamming, trolling, racism, NSFW stuff, and other forms of sucking will not be tolerated. If anyone is griefing please report abuse. Be good. Don't suck!
Contest:
7:30 PM on 07.11.2011 | 
I'm scared...
:/
Aside from that, I agree with Jim's stance on MC having a valuable role and not really being the "bad guy". In the end it does come down to the people who misuse Metacritic, I suppose.
Or to put it in another way: haters going to hate.
Back then we lived by reviews and were never wronged by them. It's sad to see them become so worthless with the general attitude that "numbers don't matter" anymore.
Agreed so hard.
And for the record, no one is worrying about you caring when it comes to Metacritic. It is about publishers using the scores to deem game failures and in some cases, avoid paying bonuses.
"Who cares? If you really like something, than a numbered score from a complete stranger shouldn't really affect you, regardless of what they say unless; you secretly agree with that person. "
Not sure if serious... but just in case you are, I'd think it should be obvious that reviews are mainly for people that haven't bought a specific title, yet, and want to know how it fares.
Destructoid keeps up the good Work.
I've learned a lot about just the ins and outs of reviewing from reading this site (mostly Jim's words on it but also Nick, Jonathan and Dale have weighed in on it from time to time too I believe) and I have to say, even without reading those things, its just common sense that reviewers can't do this.
Theres far to many games coming out from far to many companies and far too little time in the day, let alone our lives, to do achieve his goals no matter how noble they may be. And most review sites don't have the budgets to hire another 3 to 8 people to even try to attempt this.
What will stop this idea in its tracks, if it were to happen, is if reviewers gave bad reviews for games, whether they're good or not, AAA or not, because they have to rush their reviews.. Or even worse to try to continue to skew the metacritic ratings in one direction or another, even for the shittiest games out there.
I mean, we have to remember one of the primary problems is that the PR firms attacking reviewers for low scores whether the games deserved them or not.. this doesn't feel like it'll change that.
Buy, rent, pass would work if everyone thought the same way the problem with that is really the same problem that reviews have now days, they are based on opinion so while someone might say hey you should buy this game and you do, you might end up playing 20 minutes and be like umm this was a waste of money.
I think all reviews should be required to point out all the pros and cons in a manner that they don't through there opinion into it, because i know i would be more likely to avoid a title with gamebreaking glitches then a title with a few glitches.
example fallout: New Vegas couldn't beat the game because the guys that were suppose to go with me into the final fight ended up glitching in one spot and stopped moving all together, had i known that would happen i would have avoided the game.
I'm not disagreeing with you on that, as there are quite a few games that I was on the fence on that I eventually bought based on good reviews (Bioshock, Batman Arkham Aslyum, etc.). However, what I am saying is that a lot of people take reviews of games that they like way too seriously, and if someone says something even remotely negative of their favorite game, they'll get pissed off and throw a hissy fit. Remember how many people got upset at Jim when he gave the Witcher 2 a 6.0, or gave Final Fantasy 13 a 4.0? Hell remember when former Gamespot editor Jeff Gerstmann gave Twilight Princess an 8.8? That last one wasn't even a negative score, and was talking negatively about the presentation (which a lot of critics brought up in their reviews), and he STILL felt nerd rage from every Zelda fan on the Internet.
Don't get me wrong, I still think reviews are vital for people who may be on the fence when it comes to a game, but to get upset about how one person feels about a game because it's too low or too high has always been very silly to me. That's the point I was trying to make.
Dtoid does do great work, but they use the .5 on occasion.
Personally, I have no problem with it.
Jim Sterling doesn't have this issue, clearly, and more reviews should look like how he operates. Honest to goodness opinion based on how they feel about the game, not a page of technical whogivesafuck and a general pandering to what is and is not seen as popular by the masses. Who cares what is and is not considered good and bad game elements, tell me how you feel while/after playing a game. Most reviews read like theyre written by review machines, and thats certainly now how I fucking play them.
And you know what? Final Fantasy 13 was technically sound and I too would have given it a shit score. Why? Because I didn't enjoy it despite how well it was made. Objectivity is for reviewing a car, or a TV. Its value in reviewing an experience is nothing compared to the value of honest subjectivity.
Almost as if he thinks reviewers write their reviews to serve the good of Metacritic, with their own outlets (that pay them) just getting in the way. Aren't you supposed to just be a service?
If the reviewer finds a game to be as good as Half-Life, for example, he/she places it in the same tier as Half-Life (whatever that tier is called). Now, if the reviewer can with certainty say that he/she instead holds the reviewed game in higher or lower regard, then the game gets placed in a tier above or below Half-Life. That way, games that the reviewer can't separate between based on overall quality end up in the same tier. Most tiers would probably contain multiple games, but there might also be tiers containing only one (maybe the reviewer has an all-time favorite game that he/she thinks can't be touched by any other game, for example).
You'd then have a link to a list of all the games (sorted into tiers) that particular reviewer has reviewed. Better yet, it would include a bunch of games that the reviewer hasn't nessessarily reviewed but has still played and has a clear opinion of. You could then read every reviewer's list to find out which of the reviewers you resemble the most, opinionwise.
Taking inspiration from RockWallofMight359, the list might also indicate where the line is drawn for games that are buy, rent or passworthy (or alternatively simply worth the reviewer's time).
A made-up list:
TIER 1 (buy)
Half-Life 2
TIER 2 (buy)
Mother 3
Resident Evil 4
Metal Gear Solid 3
TIER 3 (rent)
Deus Ex
etc.
It wouldn't really work for Metacritic, but...
No scores, just tiers.
http://www.metacritic.com/about-metascores
Reviewers are just some random jagovvs spouting their own opinion and there's no guarantee that their taste matches yours, and I've seen so many of them directly contradict themselves. I once saw a reviewer say "on the downside, this game contains no voice acting at all" and in the very next review, they said "Jeez, who thought it was a good idea to have video game characters talk?"
Rent the game, borrow it from a friend, watch trailers and gameplay footage from real people, and make up your own damn mind!
O cool thank you kind sir.