Mathematical
Balancing, balancing, balancing. Developers do as much as they can to make sure their multiplayer is designed as best as possible before shipping, but it’s when the title has made it into the public’s hand that really puts everything to the test. Making sure a game is balanced well is a never-ending battle with any multiplayer game.
Black Ops II is no different, and Treyarch has created a new system called Black Box that analyzes player data to really see how everything is performing. “What we said was we need to balance the emotion of using something, like your personal visceral reaction to it, with math,” design director David Vonderhaar told me. “There’s a lot of instrumentation in this game, there’s a lot of data logging, so I know exactly what the power band of a weapon is at all times. So I can combine how something feels with how it’s actually behaving. I know, I know this is going to happen: A million people are going to Tweet me and tell me that the [for example] ‘PDW 57 is overpowered, Vonderhaar what are you doing?’
“What I’ll do is I can go and look at the math and I’ll go ‘Actually, you know what dude? You’re just good with it. Have fun, because the math says that it’s not overpowered.’ I have the averages, you’re being successful. That means Pick 10 is working, and if it’s overpowered I’ll know it for real. Like not because someone said it was, or because I felt it was, or because you sent me an angry Tweet because you got killed by the SRM Shotgun one night 17 times so you’re mad. The data will tell me the truth.”
“So combining your opinion with mathematical fact is the way we can handle this and that’s only possible if the game is instrumented well, and it is,” David continued. “Like a whole new level of instrumentation, it’s crazy cool. Data nerds would just die if they had any idea what kind of stuff we can yank with this.”
Basically, Black Box is a series of databases that allows Treyarch to run various queries against it in lots of interesting ways. They can take a weapon like the SRM with the long barrel attachment, for example, and compare how many shots it’s taking to put someone down on average against some other data set. That’s one example of the cross checking they can perform now.
“Once you have the data you just need to figure out how you want to use it. You can create new queries and modify queries and that allows you to run all sorts of reports. I have this chart on my wall that we run all the time to just be sure how things are trending. What’s really cool about it is you make a little tiny tweak [and] you get a sense of what that does to the tuning, what would happen if you did that. It lets you do some really lightweight modeling, it’s pretty neat.
“I’m just going to kill people with facts [with] this game when they come over and go ‘This thing is OP.’ I’m going to go ‘Actually it’s not OP, you’re wrong.’ I’m not going to yell at them or anything, I’m just going to tell them the truth. I don’t have to hide this stuff from people, just drop knowledge bombs on guys. It’s gotta be done! What people are doing is reacting emotionally, and I’m like a really emotional guy so I get it.”
David is one of the more prominent developers out there who actually uses Twitter as real-time feedback in a large way. “It’s like the world’s greatest instant fact checker. You can get really close to something when you work on it so it’s always healthy to say, look, let me just put this out in the stratosphere and see are my assumptions about it still right and that’s when you’ll see me do that. The other thing is it’s really great to just tell people look this is how it’s going work and why. You may not agree with that but I’m going to tell you. I’m going to be transparent and tell you.”
Social media also proves to be a great early warning detection system for David as well. “It’s a really good way to understand what’s happening before something becomes. That allows us to be on top of it. You can’t tell people what they want to hear, that’s the wrong way to do it. But what you can do is say ‘I got it. Thank you. I’ll look into it. I looked into it and this is what we’re doing.’ Or not, and then you’ll move on from there hopefully.”
And while he’s open to hearing everyone’s feedback, at the end of the day “you can’t let Twitter design the videogame. These are just fans, and they just want to talk to somebody. Who are they going to talk to? A guy that doesn’t work on the game? I’m the best guy for them to talk to, someone who can actually make decisions about the game is infinitely better than talking to somebody who has no control over what actually happens. I think it’s better that way, even if it comes with this whole new level of responsibility and possibly headache. It’s worth it in the end. I’m glad for it and I’m glad for them, honestly.”
