Latent Representation in Human-Robot Interaction With Explicit Consideration of Periodic Dynamics

Taisuke Kobayashi, Shingo Murata, Tetsunari Inamura

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


This article presents a new data-driven framework for analyzing periodic physical human-robot interaction (pHRI) in latent state space. The model representing pHRI is critical for elaborating human understanding and/or robot control during pHRI. Recent advancements in deep learning technology would allow us to train such a model on a dataset collected from the actual pHRI. Our framework is based on a variational recurrent neural network (VRNN), which can process time-series data generated by a pHRI. This study modifies VRNN to explicitly integrate the latent dynamics from robot to human and to distinguish it from a human state estimate module. Furthermore, to analyze periodic motions, such as walking, we integrate VRNN with a new recurrent network based on reservoir computing (RC), which has random and fixed connections between numerous neurons. By boosting RC into a complex domain, periodic behavior can be represented as phase rotation in the complex domain without decaying the amplitude. A rope rotation/swinging experiment was used to validate the proposed framework. The proposed framework, trained on the collected experiment dataset, achieved the latent state space in which variation in periodic motions can be distinguished. The best prediction accuracy of the human observations and robot actions was obtained in such a well-distinguished space.

Original languageEnglish
Pages (from-to)928-940
Number of pages13
JournalIEEE Transactions on Human-Machine Systems
Issue number5
Publication statusPublished - 2022 Oct 1


  • Complex domain
  • human-robot interaction
  • latent space extraction
  • motion analysis
  • recurrent neural networks (RNNs)

ASJC Scopus subject areas

  • Human Factors and Ergonomics
  • Control and Systems Engineering
  • Signal Processing
  • Human-Computer Interaction
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'Latent Representation in Human-Robot Interaction With Explicit Consideration of Periodic Dynamics'. Together they form a unique fingerprint.

Cite this