Multiple planes based registration using 3D Projective Space for Augmented Reality

We propose a vision-based registration method for Augmented Reality (AR) that uses real scenes containing multiple planar structures. Our method provides users with an enhanced view of a real scene by overlaying computer-generated virtual objects onto images captured with a movable camera. The virtual objects are associated with a 3D planar structure and appeared to move with that structure. To align the coordinates of the virtual objects with the coordinates of the images captured with the camera, the camera's motion should be estimated for every frame by tracking and integrating the multiple planes. In contrast with related work, the proposed method does not require a priori knowledge about the geometrical relationships among the multiple planes because the geometrical relationship is automatically estimated by "Projective Space". This space is 3D and is defined by projective reconstruction of two reference images. Automatic estimation using the Projective Space can eliminate the time-consuming task of measuring the planes and the constraints of plane arrangement, e.g. only vertical or co-planar. Therefore, the proposed method can be applied even to complicated multi-planar scenes.