3D line segment based model generation by RGB-D camera for camera pose estimation

In this paper, we propose a novel method for generating 3D line segment based model from an image sequence taken with a RGBD camera. Constructing 3D geometrical representation by 3D model is essential for model based camera pose estimation that can be performed by corresponding 2D features in images with 3D features of the captured scene. While point features are mostly used as such features for conventional camera pose estimation, we aim to use line segment features for improving the performance of the camera pose estimation. In this method, using RGB images and depth images of two continuous frames, 2D line segments from the current frame and 3D line segments from the previous frame are corresponded. The 2D-3D line segment correspondences provide camera pose of the current frame. All of 2D line segments are finally back projected to the world coordinate based on the estimated camera pose for generating 3D line segment based model of the target scene. In experiments, we confirmed that the proposed method can successfully generate line segment based models, while 3D models based on the point features often fail to successfully represent the target scene.