Virtual omnidirectional video synthesis with multiple cameras for sports training

This paper proposes a new method to synthesize an omnidirectional video at a viewpoint inside a sports ground, with the goal of sports training. If athletes could virtually experience real games from a player's viewpoint, they might possibly be able to exhibit higher performance in an actual game. A head mounted display, which makes it possible to watch intuitive and interactive omnidirectional video from a 360-degree player's view, together with head direction tracking, leads to further enhanced effectiveness of training. However, it is difficult to put an omnidirectional camera on the field during a real game. Therefore, techniques for synthesizing an omnidirectional video at a player's viewpoint (virtual viewpoint) with the cameras outside the field are required. With this aim in mind, we propose a fast and stable omnidirectional video synthesis technique with image inpainting, which removes unwanted occluders between the virtual viewpoint and the cameras.