Baby Touch: Quantifying Visual-Haptic Exploratory Behaviors in Infants of Sensory-Motor Development

We propose a method to quantify an infant’s exploratory behavior by taking a video of his/her actions and analyzing them. Exploratory behaviors of infants are known to be necessary for the development of cognitive functions and language acquisition. Multiple studies on exploratory behaviors have been conducted; however, exploratory behaviors of infants have been commonly classified and quantified manually, requiring ample efforts. Advances in computer vision research using machine learning in recent years have made it possible to automatically analyze captured videos and register movements of the human body. In this study, we developed a measurement system that enables the quantification of exploratory behaviors of infants by combining OpenPose, OpenFace, and a computer vision library. First, we created a video-capturing environment suitable for capturing an infant’s behavior. Second, we integrated the computer vision library to analyze infants fixating on and touching objects placed in front of them. As a result, we are now able to quantify some aspects of infants’ exploratory behavior. Our measurement system will be useful for investigating the exploratory behavior of six to fifteen-month old children using visual-haptic modalities, and also it will also be valuable in comprehending the developmental stages of each child.