The igf has been around for twelve years now promoting the best indie games the community has to offer, or is it? Until this year I had no idea how the judging process took place for the competition itself, but it seems that a few developers are unhappy with the system they use, specifically the first round of judging to pick the finalists.
How this round is judged seems to be that each of this years 300, or so, games is assigned around a dozen judges who give the game a score of 0-100 in each of the 5 categories. A judge can then leave notes on the game, giving their opinions on it, technical difficulties with the game etc. for other judges to see. There is a total of roughly 150 judges, who are a mixed bag of commentators and game developers, who can comment on any game, but only score on the ones they are assigned to. There was also the mandatory inclusion of anonymous feedback for the games developer.
A more in depth look can be found here:
The problem some indie developers have seems to be that there are too many judges and they are drawn from too disparate a crowd of people to be effective in judging the games with the some common perception of a "good" game, or even common perception of how to judge the competition. To further this point an indie developer, Arthur Lee(Mr. Podunkian, creator of the Merry Gear Solid series), points out that the vast majority of the judges in the competition aren't even associated with indie games, but rather are mostly mainstream figures. Complaining that the entire competition seems to be pushing the indie games that are the most mainstream styled to the forefront, through its selection of judges. Another developer with this sort of problem is one Anna Anthropy(Auntie Pixelante, creator of Jill Off and Calamity Annie), who wrote a lengthly post on her blog detailing all the problems she had with the entire process.They both share the same problems in the judgment process, both recalling when the categories were titles with "innovation in" rather than "excellence in", the first implying that the indie games were going somewhere that most mainstream games wouldn't go and rewarding these new ideas, the later rewarding what mainstream games are currently doing, refining existing ideas to achieve "excellence". Anna adds that the judges notes section seemed to be used exclusively for technical issues with the games, such as not working on OSx as opposed to what she suggests they are for, which is pointing out what was done well in the game and what was done wrong.
Its not all negativity though. Mark Rose(Games journalist and editor for indiegames.com) complains about the diversity of games he was presented with, saying he played games ranging from superb, to not-so-fantastic and that the topic of the length of the games was difficult to factor in, saying some games took hours whilst others took minutes. He also has a different opinion on the role of the judges notes in the competition, claiming that they should be used exclusively for technical issues and that the people posting their opinions were causing him to approach games with a certain bias, based on others opinions. Depite these problems he says he is overall happy with how the judging system works, claiming every game has an equal opportunity to win the competition.
Personally I think the competition is drifting away from the overall experience of indiegames. I was drawn to these games because they were different. Almost all of the indiegames i have played had some new style of game play or mechanic that I had not seen in the mainstream. I said that mainstream seems to currently be moving towards refining game play mechanics that are tried and tested. Thats not to say they aren't innovating too (see Portal, Mirrors Edge) they're just investing too much time and money into a game to try something completely radical. Thats were indiegames come in, they are relatively easy and cheap to make and so you can do something a lot more radical with them. Its just there are too many judges in the competition who are used to applauding the games with the best graphics, or best gameplay, not the ones with the most innovative gameplay mechanics that work well
They don't have to be absolutly perfect, they need to be innovative and they need to work well in the context of the game.
On the point of there being too many judges I would like to point out the following anonymous comment one developer received for his game.
"Sadly I think this game suffers from being just another puzzle platformer, which basically hurts every game that isn't Braid."
This comment was about the game Verge developed by Kyle Pulver(Zaratustara). The game was on the theme of Life and Death. When you died in the game you went to a netherworld were you could move around to places you could not get to in the real world and this was the crux of the game. This is a completely different mechanic to the one used in the puzzle platform game Braid, by Johnathan Blow, which utilizes time travel. This "feedback" reflects very poorly on the people who were selected to judge the entire competition. There needs to be some way making it clear that this is a completely inappropriate piece of "advice" to offer a developer who spent a month on a game that is their own creation. This developer paid $99 to be told that his game didn't meet the standards because "Its not Braid". That is unacceptable in a competition that is supposed to be about how these games stack up for their ideas and originality.
I don't think the competition should be abandoned for these problems though, they should just try to select judges a lot more carefully. Maybe ask the people who actually play indie games to volunteer their time to judge the games, or at least publish some stricter guidelines on what the competition expects out of its judges. The competition is only twelve years old and is still growing,
Arthur Lee's article on pigscene(A lot of irony and strongly worded dissatifaction):
Tigsource's forum expressing several of the tigsources developer's problems with some of the anonymous feedback:
Auntie Pixelante's blog entry:
IGF's official report on the judgment process:
Mike Rose's experience as a judge: