Preliminary Study of Object Recognition by Converting Physical Responses to Images in Two Dimensions

The use of robots is desired as a replacement for human labor. However, it is difficult for robots to respond flexibly to changes in objects and environments and perform tasks. Recently, many systems have been proposed that can flexibly respond to changes by generating robot motions using machine learning. Many machine learning methods use a camera to acquire environmental information, and feature extraction is performed using images acquired from the camera using CNN (Convolutional Neural Network), CAE (Convolutional Auto Encoder), or other methods. Many methods estimate the input values in the next step by inputting the image features, position data and reaction force data acquired from the robot together into the RNN (Recurrent Neural Network), etc. However, in most cases, the relationship between the image and robot data is learned without explicitly stating it. Therefore, in this paper, the data acquired from the robot is converted to images and used in combination with images from the camera to make the interaction between the robot and the environment explicit and to improve the estimation accuracy of NNs. In simulations, the proposed method was used to perform the task of discriminating the target of motion, and the high estimation accuracy was confirmed. In the future, we plan to use this method as input data for motion generation to generate motion according to the object.