Robotic palpation system - reproduction method of dermatologists' skin palpation judgment using a deep neural network

The recent surge in infectious diseases, such as COVID-19, has amplified the need for medical examinations that minimize contact between doctors and patients. This is particularly relevant for medical treatments requiring palpation, especially in dermatology. In this study, we aimed to replicate the assessment of softness and surface textures of affected areas, a critical aspect for dermatologists in diagnosing conditions, using a simple robotic device. We derived five levels of softness and three types of surface textures from 14 types of materials based on interviews with dermatologists. To elicit a haptic response from the materials during the pushing procedure, we developed 1) a single-rod probe equipped with a haptic sensor (measuring force and acceleration) using a linear actuator, and 2) a dual-rod configuration supplemented with a nearby vibrator to capture vibration propagation through the material. Frequency-analyzed images were generated from the waveforms of force and acceleration obtained. A total of 500 images from 13 different materials were evaluated for discrimination using AlexNet-based transfer learning. The discrimination accuracy for the 13 materials was 97.4 % (average across trials) when combining the force and acceleration data from the single-rod probe with the acceleration image from the dual-rod probe. The discrimination accuracy was 96.4 % using only the force and acceleration data from the single-rod probe, suggesting that adding acceleration data during vibration propagation enhances discrimination accuracy. A similar comparison was conducted for the five levels of softness and surface features, indicating that acceleration during vibration propagation may provide essential information for palpation.