Augmented classification of Japanese visemes and hierarchical weighted discrimination for visual speech recognition

For the purpose of automatic speech recognition and speech animation synthesis, speaker verification and so on, there have been studies on 'viseme'. Viseme is a visually identifiable unit of utterance or the equivalent unit in the visual domain of the phoneme in audio domain. The classification and the discrimination method of visemes are still important topics. This paper focuses on the number of classification units and a discrimination procedure of Japanese visemes: We extend the number of visemes from 6 to 9 to expanse the word representation by their series, then propose the hierarchical weighted discrimination using multiple discriminative analysis (MDA) to enhance the discriminative ability. In order to verify and discuss the availability of our proposals, visemes discrimination and word recognition experiments were conducted. From these results, the validity of the proposed methods was confirmed.