TY - GEN
T1 - ReflecTouch
T2 - 2022 CHI Conference on Human Factors in Computing Systems, CHI 2022
AU - Zhang, Xiang
AU - Ikematsu, Kaori
AU - Kato, Kunihiro
AU - Sugiura, Yuta
N1 - Publisher Copyright:
© 2022 ACM.
PY - 2022/4/29
Y1 - 2022/4/29
N2 - By sensing how a user is holding a smartphone, adaptive user interfaces are possible such as those that automatically switch the displayed content and position of graphical user interface (GUI) components following how the phone is being held. We propose ReflecTouch, a novel method for detecting how a smartphone is being held by capturing images of the smartphone screen reflected on the cornea with a built-in front camera. In these images, the areas where the user places their fingers on the screen appear as shadows, which makes it possible to estimate the grasp posture. Since most smartphones have a front camera, this method can be used regardless of the device model; in addition, no additional sensor or hardware is required. We conducted data collection experiments to verify the classification accuracy of the proposed method for six different grasp postures, and the accuracy was 85%.
AB - By sensing how a user is holding a smartphone, adaptive user interfaces are possible such as those that automatically switch the displayed content and position of graphical user interface (GUI) components following how the phone is being held. We propose ReflecTouch, a novel method for detecting how a smartphone is being held by capturing images of the smartphone screen reflected on the cornea with a built-in front camera. In these images, the areas where the user places their fingers on the screen appear as shadows, which makes it possible to estimate the grasp posture. Since most smartphones have a front camera, this method can be used regardless of the device model; in addition, no additional sensor or hardware is required. We conducted data collection experiments to verify the classification accuracy of the proposed method for six different grasp postures, and the accuracy was 85%.
KW - Corneal reflection images
KW - Hand grip detection
UR - https://www.scopus.com/pages/publications/85130550746
UR - https://www.scopus.com/pages/publications/85130550746#tab=citedBy
U2 - 10.1145/3491102.3517440
DO - 10.1145/3491102.3517440
M3 - Conference contribution
AN - SCOPUS:85130550746
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI 2022 - Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
PB - Association for Computing Machinery
Y2 - 30 April 2022 through 5 May 2022
ER -