3D head pose tracking using a particle filter with nose template matching

In this paper, we propose a method for real-time tracking of the driver's head pose in a real vehicle environment by using multiple NIR cameras. In order to achieve real-time performance and high accuracy, the head pose is estimated with six degrees of freedom by a particle filter with a restricted state space. First, the 3D position of the nostrils is measured by template matching of stereo images. Because the nostrils are the darkest area in the captured NTR images, we can detect the position of the nose robustly and very precisely. From the 3D position of the nose, we can make an initial estimate of the head pose. Then, each hypothesis is updated by prior probability with the constraint of the nose position. Thus, it is possible to reduce the number of particles while maintaining accuracy. Experimental results indicate that this method is effective for head pose tracking.