Texture overlay for virtual clothing based on PCA of silhouettes

In this paper, we propose a method for overlaying an arbitrary texture image onto a surface of a plain T-shirt worn by a user. For overlaying arbitrary textures onto the surface of the T-shirt, we need to know the deformation of the surface. For estimating the deformation of the surface from the input images, we use a two-phase process: learning and searching. In the learning phase, the system learns the relationship between the deformation of the surface and the silhouette of the T-shirt region in the image. A database of a number of training images in which a person wearing a T-shirt with markers moves through a variety of positions is used for this learning. Using the database, the system can learn the relationship between the shape of the silhouette and the surface deformation that is provided by the 2D positions of the markers on the surface of the T-shirt. In the searching phase, the silhouette of the user's T-shirt is extracted from the input image, and then, a search for a similar silhouette in the database is conducted in the subspace of the silhouette, which is computed using a PCA of the database. By using the proposed method for estimating the deformation of the surface of the T-shirt, we perform experiments for overlaying virtual clothing.