As the technology of games continues to develop, we're crossing into the Uncanny Valley more easily and with greater frequency than ever before. We've progressed beyond simply making something lifelike into the realm of photorealism, and it's getting harder all the time to discern what's an actual person and what's not. At the front of the pack for such advances is a Santa Monica company called Image Metrics. Their contribution to the upcoming Grand Theft Auto IV, according to Rockstar's Dan Houser, ripples out past simply capturing and animating lifelike facial expressions and into expanding the scope of the storytelling. Houser explains:
"In terms of the characters, as we were bringing up the quality of the graphics and the physics and the animation, the quality of the writing and story structure were always balanced to reflect those [things]...the reasons we could pull that off and make the cutscenes slightly longer is because we've got such good facial animation that you can afford to be a little bit more slow-paced in them and a little more intimate with some of them. They've got to look beautiful, some of the cut scenes, and they hold the game together nicely."
At the center of Image Metrics' magic is the ability to capture facial metrics directly from video, without all the muss and fuss and mini-marshmallows-glued-to-the-face of motion capture. Now, actors can perform unhindered for the cameras, and even their tiniest cringes and nasal flares are translated to their virtual counterpart. When applied to a character model with a facial polygon count of over 3000, the effect is simply amazing, as seen in the video demo on their homepage. Most notable is the detail with which the process captures the positions of teeth, lips, tongue and jaw. When audio is captured along with the video, lip syncing is perfectly accurate and loses nothing in translation to the virtual domain.
IM's work has been seen in games like Devil May Cry 4, Call Of Duty 2, and the recently released Unreal Tournament 3, as well as various commercials and movies. Come April 29, they'll be putting a new face on our beloved Liberty City too.
Their work will surely not go unappreciated, when I experiment with these new levels of expression and realism as I use the exquisitely detailed and realistic faces as sniper targets. BLAM! "Ooh! Did you see how unhappy that guy was?"