surf dtoid with 
Rising (10+)
People you follow
I test software and I can tell you that this is not going to work. Beta testers (especially video game consumers) are only looking out for number 1. Payed software testers are trying to find bugs that exist outside of normal playing, low traffic areas. They find incidents that don't always result in failures. What about functionality, beta testers can tell you if something works, but can they tell you if it works how it was meant to work? I am all for beta testing but you cannot cut QA out of the loop completely, I am sure they will find the out after the first release or two.
Good point. I think a more realistic statement is to say that more general style testing will be distributed, but it's hard to replace people in-house completely. Otherwise the Russian mail-order bride industry would really boom
Oh christ, like we need people at that stage acting as game critics on any level... This reminds me of all the posters you see on AICN who get into like 6 month before release film screenings and rip a film apart on it's lighting or some stupid shit.
"I really wanted to like this game, but I felt the direction the game creator was taking in Act VI was confusing. As a result, please find enclosed my revised dialog between the protagonist and a character I have renamed as Slorgoth (nee Billiam)......." and so on.
Testers were hired because they could respect an NDA, do their job, and had qualifiable experience TESTING a game. Assuming that some kid in Padukah will run through your levels trying to do everything is REALLY stretching it. You'd be lucky if anyone in beta or even alpha testing runs through a good 60% of the game elements, when they're offered a free chance to play a game early. I know my exposure to most games probably runs at 70% and I ain't drawing a paycheck to walk to every corner of every map to find holes in the landscape where my character can't walk.
"I really wanted to like this game, but I felt the direction the game creator was taking in Act VI was confusing. As a result, please find enclosed my revised dialog between the protagonist and a character I have renamed as Slorgoth (nee Billiam)......." and so on.
Testers were hired because they could respect an NDA, do their job, and had qualifiable experience TESTING a game. Assuming that some kid in Padukah will run through your levels trying to do everything is REALLY stretching it. You'd be lucky if anyone in beta or even alpha testing runs through a good 60% of the game elements, when they're offered a free chance to play a game early. I know my exposure to most games probably runs at 70% and I ain't drawing a paycheck to walk to every corner of every map to find holes in the landscape where my character can't walk.
I know my point isn't addressed in the article, but the fact remains that 90% of people out there DO NOT think testing is what it really is... I'd bet they think it's glorfied critcal evaluation over the actual game content rather than QC.
Yeah, I think everyone's really hitting the nail on the head. Automated testing is handling the brainless cases of testing (checking items, walkable areas, etc). All the boring shit no human needs to test. It's now more than ever that you need trained, skilled testers who can do analytical thinking on the higher level issues of your game. It's not the monkey job that most people think. Games can really suffer when your testing team is not up to snuff and you're forced to release bug-ridden titles.
I agree with the first response. Loved to see how they handle the technical requirements. Didn't Acclaim go out of business? *shrug*
"Yeah I mean why do we want to pay beta testers, if kids PAY to beta test it themselves?! we've been doing that for years!"
"er.. thats not going on tape right?"
"er.. thats not going on tape right?"
This is how many people enter the game industry and how they gain experience. I was hoping to become a full time QA person when I graduate college and make my way up the ladder, but I hope automated testing doesn't take over.
ok. back to reality.
EPIC (Gears Of War) has said the exact OPPOSITE of what aclaim proposed, they actually DESIGNED the game by beta testing it.(Design by iteration). After playing a beta the designers or beta testers would go, "wouldnt it be cool if we added this?" then added it to see how it worked. Thats completely IMPOSSIBLE using the "user beta test" system, the development would drag forever+ if each (and most probably stupid) feature made up by users were implemented and tested.
Since EPIC won the Game of the year award just about everywhere, and Acclaim is well.. barely alive after bankruptcy, I think theres more than a slight chance developers will listen to EPIC instead.
kthxbye!
EPIC (Gears Of War) has said the exact OPPOSITE of what aclaim proposed, they actually DESIGNED the game by beta testing it.(Design by iteration). After playing a beta the designers or beta testers would go, "wouldnt it be cool if we added this?" then added it to see how it worked. Thats completely IMPOSSIBLE using the "user beta test" system, the development would drag forever+ if each (and most probably stupid) feature made up by users were implemented and tested.
Since EPIC won the Game of the year award just about everywhere, and Acclaim is well.. barely alive after bankruptcy, I think theres more than a slight chance developers will listen to EPIC instead.
kthxbye!
I had to Wiki Acclaim because I read an article this morning talking about its PR blunders in 2004 and its subsequent bankruptcy and virtual annihilation. That company was Acclaim Entertainment. I'm assuming the Mr. Perry you speak of works for Acclaim Games, which is the new and improved Acclaim run by a guy from Activision. It makes free MMORPG's. Confirmation?
This is the same guy that drove Shiny into the ground after a series of crap games. Testing has to be done in a methodical and careful fashion in order to catch everything, and I doubt you're gonna get any unpaid kids to do that.
Anything outside the stress tests of MMOs, this idea is dead before it can hit the floor. Thanks Mr. Perry, just shut up and make more Earthworm Jim games. GOOD ones.
Anything outside the stress tests of MMOs, this idea is dead before it can hit the floor. Thanks Mr. Perry, just shut up and make more Earthworm Jim games. GOOD ones.
Wow, good luck with that Acclaim. I'm a QA tester with another publisher and I must say, getting paid to test sucks enough, but doing it for free!?! no way...
Most people will probably download a build just to check it out, and then they will see hundreds of bugs and be turned off of the game completely. And for the people that actually stick it out and test through the entire QA process, I guarantee they will not be willing to pay to play that game at release (and it would be very sh*tty if Acclaim didn't provide them with a free copy anyway).
However, it's completely different if they are paying people to test from home. That would be sweet.
Most people will probably download a build just to check it out, and then they will see hundreds of bugs and be turned off of the game completely. And for the people that actually stick it out and test through the entire QA process, I guarantee they will not be willing to pay to play that game at release (and it would be very sh*tty if Acclaim didn't provide them with a free copy anyway).
However, it's completely different if they are paying people to test from home. That would be sweet.
I work in QA myself and he's pretty much talking right out of his arse. As mentioned above, I'm all for getting the game buying public involved with multiplayer beta builds and such, but surely he must understand how much work goes into a game before it even gets a sniff of beta status.
And all that about bots doing the testing. What a fucking joke! If you've Ai that good put it in the game.
And all that about bots doing the testing. What a fucking joke! If you've Ai that good put it in the game.
I believe that Automated Testing is mostly just a magical word pitched at management to get them thinking they can save money. It sounds easier than it is. Even if a company can automate 75% of the test cases that need to be run (75% is a hard number to reach) you still need to hire qualified engineers to write and maintain those automated scripts, plus the original test cases have to be written by a qualified tester to ensure that you are covering as much of the program as possible. QA is an in depth job that requires knowledge, skill, and experience. No beta program can cover what QA can do. Cutting the QA department from their company is what I have had described to me as a slow poison, it takes time before they will realize why they are dying.
I always wondered why they didn't set up dev testers at their own homes. Have them participate from their own houses, with their own PCs. They only get paid if they meet their requirements (so you can convince them not to slack, order pizza and watch Babylon 5 all day) and in that manner you'll get a broad spectrum of hardware testing as well, from lower-end systems to the newest and speediest rigs. You cut the whole office, food and housing completely out of the equation.
"You cut the whole office, food and housing completely out of the equation."
How do you figure? OK so you have more office space but they are still paying for an office, the developers and management have to be somewhere. Food and Housing? I am not even sure what this means, I haven't heard of companies paying for food, other than some snacks in a break room, and I have never heard of employers paying for housing.
"They only get paid if they meet their requirements"
Requirements in testing is hard to gauge because you can never fully test a product, so if you say something like the tester will only get paid after finding 75 bugs or after coverage of a certain percentage of test cases what incentive is there to go above and beyond if you are working from home?
"you'll get a broad spectrum of hardware testing as well, from lower-end systems to the newest and speediest rigs"
This is what Beta Testing is for, they give you the real world scenario on a multitude of systems, but you use the in house QA team to verify that the software works on a recommended machine and that you cover all test cases written. If it is left completely to Beta testers less than half of the software will likely be tested. Bugs are constantly found in software that most customers don't come across in day to day usage. It sounds like those bugs don't matter but eventually those bugs almost always bite you on the ass. One reviewer can come across that bug and rip the whole product because of it, I have seen it happen.
How do you figure? OK so you have more office space but they are still paying for an office, the developers and management have to be somewhere. Food and Housing? I am not even sure what this means, I haven't heard of companies paying for food, other than some snacks in a break room, and I have never heard of employers paying for housing.
"They only get paid if they meet their requirements"
Requirements in testing is hard to gauge because you can never fully test a product, so if you say something like the tester will only get paid after finding 75 bugs or after coverage of a certain percentage of test cases what incentive is there to go above and beyond if you are working from home?
"you'll get a broad spectrum of hardware testing as well, from lower-end systems to the newest and speediest rigs"
This is what Beta Testing is for, they give you the real world scenario on a multitude of systems, but you use the in house QA team to verify that the software works on a recommended machine and that you cover all test cases written. If it is left completely to Beta testers less than half of the software will likely be tested. Bugs are constantly found in software that most customers don't come across in day to day usage. It sounds like those bugs don't matter but eventually those bugs almost always bite you on the ass. One reviewer can come across that bug and rip the whole product because of it, I have seen it happen.
Traditional gamers: "Acclaim is dead."
No acclaim, I wont to beta test and I wont beta test unpaid. Thanks for sharing the info so I wont buy any game from you in the future.
No acclaim, I wont to beta test and I wont beta test unpaid. Thanks for sharing the info so I wont buy any game from you in the future.
I agree with Azrael somewhat, I will stay away from products that have not been tested by a qualified QA Engineer.
Excellent read, I just passed this onto a colleague who was doing a little research on that. And he actually bought me lunch because I found it for him smile So let me rephrase that: Thanks for lunch!
birthday wishes quotes
birthday wishes quotes
follow
