Experimenting with motion tracking technology is hardly new for Microsoft. The company incorporated motion tracking with the Xbox 360 gaming console’s Kinect camera, refined it with the newer Xbox One’s Kinect and has recently been seen using similar technology with their impressive HoloLens virtual reality headset.
Motion tracking is something Microsoft is continually improving upon and Microsoft Research has just revealed some information on some of their most recent developments regarding detection and reading of hand gestures with a new system they’re calling Handpose.
The new real-time hand tracking system is based on a single depth camera and allows greater recognition of a user’s hand in different lighting scenarios and distances from the camera. Handpose appears to provide a more sophisticated method for reading subtle hand gestures and apparently can detect hands without the camera first having to detect the user’s surroundings or relative position.
We present a new real-time hand tracking system based on a single depth camera. The system can accurately reconstruct complex hand poses across a variety of subjects. It also allows for robust tracking, rapidly recovering from any temporary failures.
Most uniquely, our tracker is highly flexible, dramatically improving upon previous approaches which have focused on front-facing close-range scenarios. This flexibility opens up new possibilities for human-computer interaction with examples including tracking at distances from tens of centimeters through to several meters (for controlling the TV at a distance), supporting tracking using a moving depth camera (for mobile scenarios), and arbitrary camera placements (for VR headsets).
These features are achieved through a new pipeline that combines a multi-layered discriminative reinitialization strategy for per-frame pose estimation, followed by a generative model-fitting stage. We provide extensive technical details and a detailed qualitative and quantitative analysis.
Improved hand gesture detection is something that’s going to be more and more important moving forward, especially if Microsoft wishes to deliver on its initial promises with HoloLens which they’ve shown being used to allow people to create and edit detailed constructs in a real-time three dimensional space and could be implemented in future mobile phones and camera enabled PCs. Handpose could also be used to improve the existing motion tracking technology on the Xbox One’s Kinect which despite being significantly more advanced than its predecessor, is still quite buggy when it comes to reading hand gestures and can completely misread actions or even mistake other body parts such as a knee as a hand on occasion.
Does this make you excited for the future of motion tracking and would it ever be something you could see yourself using? Share your thoughts in the comments below.
Further reading: HoloLens, Kinect, Microsoft, Microsoft Research, Xbox One
