Video-based tracking of user's motion for augmented desk interface

This paper presents an overview of our project on an augmented desk interface system called the EnhancedDesk. The EnhancedDesk is equipped with an infrared camera for tracking a users' hand motions, a color video camera for recognizing objects, and projectors for displaying various kinds of digital information onto the desk. The key technical innovations of the EnhancedDesk include: fast and accurate tracking of multiple fingers for direct manipulation of both real and projected objects with bare hands, interactive object registration and recognition with hand gestures, vision-based tracking of a user's gaze, and addition of interactive functionality overlaid on the desk. This paper presents a brief description of the components, as well as some prototypes of applications of our augmented desk interface system.