Efficient object-oriented semantic mapping with object detector

Yoshikatsu Nakajima, Hideo Saito

Research output: Contribution to journalArticlepeer-review

30 Citations (Scopus)


Incrementally, building a 3D map in which object instances are semantically annotated has a wide range of applications, including scene understanding, human-robot interactions, and simultaneous localization and mapping extensions. Although researchers are developing efficient and accurate systems, these methods still face a critical issue: Real-time processing, because the task requires a series of heavy processing components, e.g., camera pose estimation, 3D map reconstruction, and especially recognition. In this paper, we propose a novel object-oriented semantic mapping approach aiming at overcoming such issues by introducing highly accurate object-oriented semantic scene reconstruction in real time. For high efficiency, the proposed method employs a fast and scalable object detection algorithm for exploiting semantic information from the incoming frames. These outputs are integrated into geometric regions of the 3D map, which are carried by the geometric-based incremental segmentation method. The strategy of assigning class probabilities to each segmented region, not each element (e.g., surfels and voxels), notably reduces the computational cost, as well as the memory footprint. In addition to efficiency, by geometrically segmenting the 3D map first, clear boundaries between objects appear. We complementarily improve the geometric-based segmentation results beyond the geometric only to the semantic-aware representation. We validate the proposed method's accuracy and computational efficiency through experiments in a common office scene.

Original languageEnglish
Article number8579143
Pages (from-to)3206-3213
Number of pages8
JournalIEEE Access
Publication statusPublished - 2019


  • SLAM
  • Semantic mapping
  • object detection
  • segmentation

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering


Dive into the research topics of 'Efficient object-oriented semantic mapping with object detector'. Together they form a unique fingerprint.

Cite this