TY - GEN
T1 - Digital Full-Face Mask Display with Expression Recognition using Embedded Photo Reflective Sensor Arrays
AU - Takegawa, Yoshinari
AU - Tokuda, Yutaka
AU - Umezawa, Akino
AU - Suzuki, Katsuhiro
AU - Masai, Katsutoshi
AU - Sugiura, Yuta
AU - Sugimoto, Maki
AU - Plasencia, Diego Martinez
AU - Subramanian, Sriram
AU - Hirata, Keiji
N1 - Funding Information:
This work was supported by JSPS KAKENHI Grant Number 19H04157.
Publisher Copyright:
© 2020 IEEE.
PY - 2020/11
Y1 - 2020/11
N2 - This paper presents a thin digital full-face mask display that can reflect an entire facial expression of a user onto an avatar to support augmented face-to-face communication in real environments. Although camera-based facial expression recognition technology has enabled people to augment their faces with avatars, application was limited to face-to-face communication in virtual environments. To enable digital facial augmentation with an avatar in a real space, we propose a digital face mask display system that integrates a lightweight flexible display with a thin facial expression recognition system. The thin wearable facial expression recognition system was implemented with photo reflective sensor arrays which can measure facial expressions at 40 feature points distributed across an entire face. We investigated a ten-class facial expression identification model based on an SVM training algorithm. The trained model achieved an average accuracy of 79% when identifying the facial expressions of multiple users. User experiments indicated that the proposed thin digital full-face mask display allows the wearer to control the facial expression of the avatar with a fast response rate and create a positive sense of self-agency and self-ownership toward the augmented avatar face.
AB - This paper presents a thin digital full-face mask display that can reflect an entire facial expression of a user onto an avatar to support augmented face-to-face communication in real environments. Although camera-based facial expression recognition technology has enabled people to augment their faces with avatars, application was limited to face-to-face communication in virtual environments. To enable digital facial augmentation with an avatar in a real space, we propose a digital face mask display system that integrates a lightweight flexible display with a thin facial expression recognition system. The thin wearable facial expression recognition system was implemented with photo reflective sensor arrays which can measure facial expressions at 40 feature points distributed across an entire face. We investigated a ten-class facial expression identification model based on an SVM training algorithm. The trained model achieved an average accuracy of 79% when identifying the facial expressions of multiple users. User experiments indicated that the proposed thin digital full-face mask display allows the wearer to control the facial expression of the avatar with a fast response rate and create a positive sense of self-agency and self-ownership toward the augmented avatar face.
KW - Human-centered computing
KW - Human-centered computing
KW - Treemaps
KW - Visualization
KW - Visualization
KW - Visualization design and evaluation methods
KW - Visualization techniques
UR - http://www.scopus.com/inward/record.url?scp=85099319434&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85099319434&partnerID=8YFLogxK
U2 - 10.1109/ISMAR50242.2020.00030
DO - 10.1109/ISMAR50242.2020.00030
M3 - Conference contribution
AN - SCOPUS:85099319434
T3 - Proceedings - 2020 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2020
SP - 101
EP - 108
BT - Proceedings - 2020 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 19th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2020
Y2 - 9 November 2020 through 13 November 2020
ER -