Illumination estimation and relighting using an RGB-D camera

In this paper, we propose a relighting system combined with an illumination estimation method using RGBD camera. Relighting techniques can achieve the photometric registration of composite images. They often need illumination environments of the scene which include a target object and the background scene. Some relighting methods obtain the illumination environments beforehand. In this case, they cannot be used under the unknown dynamic illumination environment. Some on-line illumination estimation methods need light probes which can be invade the scene geometry. In our method, the illumination environment is estimated from pixel intensity, normal map and surface reflectance based on inverse rendering in on-line processing. The normal map of the arbitrary object which is used in the illumination estimation part and the relighting part is calculated from the denoised depth image on each frame. Relighting is achieved by calculating the ratio for the estimated Illumination environment of the each scene. Thus our implementation can be used for dynamic illumination or a dynamic object.