Video synthesis of a tennis player's viewpoint from multiple view videos

We propose a new method for synthesizing player viewpoint images from multiple view tennis videos. Our method uses two key techniques: virtual-view synthesis and player's viewpoint estimation. In the former, we divide the object tennis scene into sub-regions, which are the background, tennis court ground, players, and ball. For each sub-region, a virtual viewpoint image is synthesized by considering the geometrical conditions of each region. Then the virtual viewpoint images of the sub-regions are merged into a virtual viewpoint image of the whole scene. In virtual viewpoint image generation, view interpolation, which is a way of synthesizing images at an intermediate viewpoint to two views of the real object, restricts the viewpoint to a position between the two views. To avoid this restriction, we propose our first key technique, whereby we are able to flexibly position the viewpoint. In our second technique, the viewpoint is computed using epipolar geometry from the center of gravity of a player. By applying the computed player's viewpoint to the former technique, we can synthesize player viewpoint images. Experimental results demonstrate that the proposed method can successfully provide a video of the tennis player's viewpoint from multiple-view video images.