Predicting and attending to damaging collisions for placing everyday objects in photo-realistic simulations

Aly Magassouba, Komei Sugiura, Angelica Nakayama, Tsubasa Hirakawa, Takayoshi Yamashita, Hironobu Fujiyoshi, Hisashi Kawai

Research output: Contribution to journalArticlepeer-review

5 Citations (Scopus)


Placing objects is a fundamental task for domestic service robots (DSRs). Thus, inferring the collision-risk before a placing motion is crucial for achieving the requested task. This problem is particularly challenging because it is necessary to predict what happens if an object is placed in a cluttered designated area. We show that a rule-based approach that uses plane detection, to detect free areas, performs poorly. To address this, we develop PonNet, which has multimodal attention branches and a self-attention mechanism to predict damaging collisions, based on RGBD images. Our method can visualize the risk of damaging collisions, which is convenient because it enables the user to understand the risk. For this purpose, we build and publish an original dataset that contains 12,000 photo-realistic images of specific placing areas, with daily life objects, in home environments. The experimental results show that our approach improves accuracy compared with the baseline methods.

Original languageEnglish
Pages (from-to)787-799
Number of pages13
JournalAdvanced Robotics
Issue number12
Publication statusPublished - 2021


  • Attention branch network
  • domestic service robots
  • photo-realistic simulation
  • physical inference

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Human-Computer Interaction
  • Hardware and Architecture
  • Computer Science Applications


Dive into the research topics of 'Predicting and attending to damaging collisions for placing everyday objects in photo-realistic simulations'. Together they form a unique fingerprint.

Cite this