Depth interpolation using tangent planes on superpixels of a color image

This paper describes a method for accurately interpolating a low-resolution depth image using a high-resolution color image. In our method, first, tangent planes on each superpixel are estimated from the sparse depth information and dense color information. Then, the neighboring superpixels that have smooth-connectable tangent planes are connected, and the image segmentation to smooth surfaces are achieved. Finally, the low-resolution depth image is interpolated using this smooth surface segmentation. In experiments with images from the Middlebury stereo datasets, our method interpolates each image at a high rate, and achieves the lowest error when compared to existing techniques.