games  anime  |  toys
Destructoid is gaming news, community, videos, and sometimes love. Take the tour or jump in with Facebook:

 


Acclaim: traditional game testing is dead photo

Nevermind robots and guys in third world countries stealing your game testing jobs -- the competition might soon include that snotty little 11 year old down the street. Acclaim's (lead guitarist?) David Perry is raising eyebrows with their distributed approach to game testing; leveraging the power of the internet to use their best customers as lab rats instead of ponying up for yuppie sweatshops (no offense, San Francisco):

"Normally, you pay 20 or 30 people to test a game for six months. You give them office space. You buy computers for them. It’s a huge cost. Instead, we decided to include the community every single chance we get, so all the testing is done by consumers. They test everything 100%.” This comes at a time when auto-testing is becoming more fashionable. Perry says, “With advanced self testing, the games play themselves. With automated testing the bot will try to go in every possible direction and in every room, every day, in every part of your game, trying to get up through the ceilings and everything. You want consumers who don’t know where to go, banging into every object, falling over everything, trying to get up through the walls. Traditional testers hammering away on something? I don’t know. Those days are going to go away.”

Seems like a good deal for Acclaim and the gaming community. They save money, and average joes get to be more involved in the industry by trying defective games before anyone else. Open betas have certainly become trendy, so it should be no surprise to anyone that these efforts are becoming more sophisticated and involved with development teams. With all the horror stories we've read from awful living conditions and hours at some sgios, I'm sure some testers will be happy to know they can pull their 20 hour fart-filled shower-starved gaming shifts from the smells and comforts of their own beds. As long as mom's paying the rent, this is good news.

[Full article on Next-Gen]


Continue: More Technology stories





prev next

21 comments | showing # 1 to 21

Barbara's Avatar
Barbara at 03/20/2007 13:48
I'd like to see how these open beta testers will help Acclaim pass the certification process - never mind TCR/TRC's across several platforms.

"What do you mean we faield because it doesn't say 'Press Start BUTTON' Press Start should be good enough, right?"
fatsarmstrong's Avatar
fatsarmstrong at 03/20/2007 14:09
I test software and I can tell you that this is not going to work. Beta testers (especially video game consumers) are only looking out for number 1. Payed software testers are trying to find bugs that exist outside of normal playing, low traffic areas. They find incidents that don't always result in failures. What about functionality, beta testers can tell you if something works, but can they tell you if it works how it was meant to work? I am all for beta testing but you cannot cut QA out of the loop completely, I am sure they will find the out after the first release or two.
Niero's Avatar
Niero at 03/20/2007 14:12
Good point. I think a more realistic statement is to say that more general style testing will be distributed, but it's hard to replace people in-house completely. Otherwise the Russian mail-order bride industry would really boom
BluDesign's Avatar
BluDesign at 03/20/2007 14:13
Oh christ, like we need people at that stage acting as game critics on any level... This reminds me of all the posters you see on AICN who get into like 6 month before release film screenings and rip a film apart on it's lighting or some stupid shit.


"I really wanted to like this game, but I felt the direction the game creator was taking in Act VI was confusing. As a result, please find enclosed my revised dialog between the protagonist and a character I have renamed as Slorgoth (nee Billiam)......." and so on.

Testers were hired because they could respect an NDA, do their job, and had qualifiable experience TESTING a game. Assuming that some kid in Padukah will run through your levels trying to do everything is REALLY stretching it. You'd be lucky if anyone in beta or even alpha testing runs through a good 60% of the game elements, when they're offered a free chance to play a game early. I know my exposure to most games probably runs at 70% and I ain't drawing a paycheck to walk to every corner of every map to find holes in the landscape where my character can't walk.
skankerzero's Avatar
skankerzero at 03/20/2007 14:14
this won't work.
BluDesign's Avatar
BluDesign at 03/20/2007 14:17
I know my point isn't addressed in the article, but the fact remains that 90% of people out there DO NOT think testing is what it really is... I'd bet they think it's glorfied critcal evaluation over the actual game content rather than QC.
retailsails's Avatar
retailsails at 03/20/2007 14:20
Yeah, I think everyone's really hitting the nail on the head. Automated testing is handling the brainless cases of testing (checking items, walkable areas, etc). All the boring shit no human needs to test. It's now more than ever that you need trained, skilled testers who can do analytical thinking on the higher level issues of your game. It's not the monkey job that most people think. Games can really suffer when your testing team is not up to snuff and you're forced to release bug-ridden titles.
Kalakaua's Avatar
Kalakaua at 03/20/2007 14:35
I agree with the first response. Loved to see how they handle the technical requirements. Didn't Acclaim go out of business? *shrug*
Azrael's Avatar
Azrael at 03/20/2007 14:51
"Yeah I mean why do we want to pay beta testers, if kids PAY to beta test it themselves?! we've been doing that for years!"

"er.. thats not going on tape right?"

ghnvt's Avatar
ghnvt at 03/20/2007 14:51
This is how many people enter the game industry and how they gain experience. I was hoping to become a full time QA person when I graduate college and make my way up the ladder, but I hope automated testing doesn't take over.
Azrael's Avatar
Azrael at 03/20/2007 15:14
ok. back to reality.

EPIC (Gears Of War) has said the exact OPPOSITE of what aclaim proposed, they actually DESIGNED the game by beta testing it.(Design by iteration). After playing a beta the designers or beta testers would go, "wouldnt it be cool if we added this?" then added it to see how it worked. Thats completely IMPOSSIBLE using the "user beta test" system, the development would drag forever+ if each (and most probably stupid) feature made up by users were implemented and tested.

Since EPIC won the Game of the year award just about everywhere, and Acclaim is well.. barely alive after bankruptcy, I think theres more than a slight chance developers will listen to EPIC instead.

kthxbye!
Hipple's Avatar
Hipple at 03/20/2007 15:22
I had to Wiki Acclaim because I read an article this morning talking about its PR blunders in 2004 and its subsequent bankruptcy and virtual annihilation. That company was Acclaim Entertainment. I'm assuming the Mr. Perry you speak of works for Acclaim Games, which is the new and improved Acclaim run by a guy from Activision. It makes free MMORPG's. Confirmation?
Farktoid's Avatar
Farktoid at 03/20/2007 15:37
This is the same guy that drove Shiny into the ground after a series of crap games. Testing has to be done in a methodical and careful fashion in order to catch everything, and I doubt you're gonna get any unpaid kids to do that.

Anything outside the stress tests of MMOs, this idea is dead before it can hit the floor. Thanks Mr. Perry, just shut up and make more Earthworm Jim games. GOOD ones.
Volcanon's Avatar
Volcanon at 03/20/2007 15:44
Wow, good luck with that Acclaim. I'm a QA tester with another publisher and I must say, getting paid to test sucks enough, but doing it for free!?! no way...

Most people will probably download a build just to check it out, and then they will see hundreds of bugs and be turned off of the game completely. And for the people that actually stick it out and test through the entire QA process, I guarantee they will not be willing to pay to play that game at release (and it would be very sh*tty if Acclaim didn't provide them with a free copy anyway).

However, it's completely different if they are paying people to test from home. That would be sweet.
relik's Avatar
relik at 03/20/2007 15:50
I work in QA myself and he's pretty much talking right out of his arse. As mentioned above, I'm all for getting the game buying public involved with multiplayer beta builds and such, but surely he must understand how much work goes into a game before it even gets a sniff of beta status.

And all that about bots doing the testing. What a fucking joke! If you've Ai that good put it in the game.
fatsarmstrong's Avatar
fatsarmstrong at 03/20/2007 16:42
I believe that Automated Testing is mostly just a magical word pitched at management to get them thinking they can save money. It sounds easier than it is. Even if a company can automate 75% of the test cases that need to be run (75% is a hard number to reach) you still need to hire qualified engineers to write and maintain those automated scripts, plus the original test cases have to be written by a qualified tester to ensure that you are covering as much of the program as possible. QA is an in depth job that requires knowledge, skill, and experience. No beta program can cover what QA can do. Cutting the QA department from their company is what I have had described to me as a slow poison, it takes time before they will realize why they are dying.
PetiePalo's Avatar
PetiePalo at 03/21/2007 08:33
I always wondered why they didn't set up dev testers at their own homes. Have them participate from their own houses, with their own PCs. They only get paid if they meet their requirements (so you can convince them not to slack, order pizza and watch Babylon 5 all day) and in that manner you'll get a broad spectrum of hardware testing as well, from lower-end systems to the newest and speediest rigs. You cut the whole office, food and housing completely out of the equation.
fatsarmstrong's Avatar
fatsarmstrong at 03/21/2007 11:39
"You cut the whole office, food and housing completely out of the equation."

How do you figure? OK so you have more office space but they are still paying for an office, the developers and management have to be somewhere. Food and Housing? I am not even sure what this means, I haven't heard of companies paying for food, other than some snacks in a break room, and I have never heard of employers paying for housing.

"They only get paid if they meet their requirements"

Requirements in testing is hard to gauge because you can never fully test a product, so if you say something like the tester will only get paid after finding 75 bugs or after coverage of a certain percentage of test cases what incentive is there to go above and beyond if you are working from home?

"you'll get a broad spectrum of hardware testing as well, from lower-end systems to the newest and speediest rigs"

This is what Beta Testing is for, they give you the real world scenario on a multitude of systems, but you use the in house QA team to verify that the software works on a recommended machine and that you cover all test cases written. If it is left completely to Beta testers less than half of the software will likely be tested. Bugs are constantly found in software that most customers don't come across in day to day usage. It sounds like those bugs don't matter but eventually those bugs almost always bite you on the ass. One reviewer can come across that bug and rip the whole product because of it, I have seen it happen.
Azrael's Avatar
Azrael at 03/21/2007 15:00
Traditional gamers: "Acclaim is dead."

No acclaim, I wont to beta test and I wont beta test unpaid. Thanks for sharing the info so I wont buy any game from you in the future.



fatsarmstrong's Avatar
fatsarmstrong at 03/21/2007 17:13
I agree with Azrael somewhat, I will stay away from products that have not been tested by a qualified QA Engineer.
hajhon's Avatar
hajhon at 07/05/2007 17:33
i love beta testing
prev next

Comment with Facebook





Click connect and comment instantly!

Comment with Dtoid





New? SIGN UP - it takes 5 seconds

Comments policy

Destructoid is an open discussion community. You don't need to "audition" to post a comment - just speak your mind. We respect differing opinions on the site, so have at it. Be smart, funny, insightful, clueless, or cute -- but back it up with substance. Keep your cool, keep it fun. We only ask that you act respectfully and above all: don't be a troll and ruin it for everyone else. Don't bring down gamers or we'll, you know, gently shoot you in the face and stuff you into a flaming mailbox. Each comment is your opportuntity to make this community awesomer. Is that even a word?

Avoiding the banhammer only requires common sense: spamming, trolling, racism, NSFW stuff, and other forms of sucking will not be tolerated. If anyone is griefing please report abuse. Be good. Don't suck!

 
New on Destructoid.TV play all videos

Loading
Loading Destructoid Videos




    Win this!
    Reminder: We're giving away six copies of Magnacarta 2!



    Dtoid Twitter    Got news?   tips@destructoid.com

    Reviews & Previews
    Mahjongg Artifacts 2 review
    Dragon Age: Origins review
    Lost Winds: The Winter of the Melodias review
    Osmos review
    Space Invaders Extreme 2 review
    Half-Minute Hero review
    JU-ON: The Grudge review
    Kenka Bancho: Badass Rumble review
    Thexder Neo review
    Domino Rally review
    more reviews
    PS3's 256-player MAG
    Rooms The Main Building
    Skate 3
    Hudson's bringing back the Bonk
    James Cameron's Avatar
    Bomberman Battlefest
    Calling
    Bad Company 2's multiplayer
    Partying like it's 1959 in BioShock 2's multiplayer
    BioShock 2 through the eyes of Big Daddy
    more previews


    - The Dtoid Army is 49642 strong -

    Showing Cblogs with 3+ faps   show all

    Call for entries: do the wrong thing

    New to Dtoid? Read the survival guide




     Originals
    Jim Sterling: How to respond to a videogame review





















    More Destructoid Originals




     Popular now more
























    Destructoid's editorial lovefest is:
    Nick Chester
    Editor-in-Chief
    Jim Sterling
    Reviews Editor
    Dale North
    News Editor
    Hamza Aziz
    Community Manager
    Anthony Burch
    Features Editor
    Rey Gutierrez
    Video editor & director
    Niero
    Founder, publisher
    Letters to the editors
    tips@destructoid.com
    Associate Editors
    Ashley Davis Jonathan Holmes
    Brad Nicholson Jonathan Ross
    Brad Rice Jordan Devore
    Chad Concelmo Matthew Razak
    Colette Bennett Tom Fronczak
    Conrad Zimmerman Topher Cantler
    Dyson Samit Sarkar
    Contributors
    Adam Dork
    Ben Perlee
    Daniel Lingen
    Joseph Leray
    Joe Burling
    Mikey
    Will Maddock
    Stella Wong





     

     
      get involved

    register or login
    post a blog
    post a forum
    enter a contest
    contribute a news tip
    suggest a feature
    be a guest editor
    support

    new member's guide
    login assistance
    tech support
    report abuse
    email our editors
    read our dev blog
    nuclear crisis?
    keep in touch

    RSS feed
    Twitter
    Facebook
    Myspace
    Flickr
    Game nights
    Meetup+play online
    seriously

    about Destructoid
    advertising
    terms of use
    privacy policy
    jobs at MM
    buy our crap
    our network

    Tomopop
    Japanator
    Despingation?




    Destructoid is an independently-run publication forged by our love of video games and the gaming community's need of accountable enthusiast press
    living the dream since March 16, 2006