Microsoft Demonstrates Highly Detailed Kinect Avatar Prototype

With around 10 million Kinect sensors now shipped, Microsoft's research arm is demonstrating a highly detailed 3D avatar that the company says could be used to create nearly photorealistic, real-time representations of users. Microsoft Chief Research and Strategy Officer Craig Mundie showed the technology to USA Today, first demonstrating a high-polygon, 3D modeled face (pictured) accurately lip-syncing to synthesized words from a typed script. "Theres no reason that we couldn't do that in real time by feeding in the information we get from a Kinect sensor, including its audio input and its 3D modeling spatial representation," Mundie said. "[We could] couple that the the body and the gesture recognition in order to create a full body avatar that has photorealistic features and full face animation." Last month, Microsoft unveiled Avatar Kinect, an upcoming Xbox Live service that will let users converse by using the Kinect sensor to control body movement and basic facial expressions on cartoon-like Xbox Live avatars. But Mundie sees a combination of 3D modeling and 2D video allowing the creation of "an avatar whose facial animation is very very realistic and the quality is almost like a photograph of a person. This is a way to create a synthetic model of people that will be acceptable to them when they look at them on television or in an Avatar Kinect kind of scenario." Other Microsoft researchers are working on using Kinect to create a 3D model of a living room's furniture, Mundie said, though both projects are still far from becoming sellable products.