HySAR: Hybrid material rendering by an optical see-through head-mounted display with spatial augmented reality projection

Takumi Hamasaki, Yuta Itoh, Yuichi Hiroi, Daisuke Iwai, Maki Sugimoto

Research output: Contribution to journalArticlepeer-review

12 Citations (Scopus)


Spatial augmented reality (SAR) pursues realism in rendering materials and objects. To advance this goal, we propose a hybrid SAR (HySAR) that combines a projector with optical see-through head-mounted displays (OST-HMD). In an ordinary SAR scenario with co-located viewers, the viewers perceive the same virtual material on physical surfaces. In general, the material consists of two components: a view-independent (VI) component such as diffuse reflection, and a view-dependent (VD) component such as specular reflection. The VI component is static over viewpoints, whereas the VD should change for each viewpoint even if a projector can simulate only one viewpoint at one time. In HySAR, a projector only renders the static VI components. In addition, the OST-HMD renders the dynamic VD components according to the viewer's current viewpoint. Unlike conventional SAR, the HySAR concept theoretically allows an unlimited number of co-located viewers to see the correct material over different viewpoints. Furthermore, the combination enhances the total dynamic range, the maximum intensity, and the resolution of perceived materials. With proof-of-concept systems, we demonstrate HySAR both qualitatively and quantitatively with real objects. First, we demonstrate HySAR by rendering synthetic material properties on a real object from different viewpoints. Our quantitative evaluation shows that our system increases the dynamic range by 2.24 times and the maximum intensity by 2.12 times compared to an ordinary SAR system. Second, we replicate the material properties of a real object by SAR and HySAR, and show that HySAR outperforms SAR in rendering VD specular components.

Original languageEnglish
Pages (from-to)1457-1466
Number of pages10
JournalIEEE Transactions on Visualization and Computer Graphics
Issue number4
Publication statusPublished - 2018 Apr


  • Hybrid material rendering
  • Optical see-through displays
  • Spatial augmented reality

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'HySAR: Hybrid material rendering by an optical see-through head-mounted display with spatial augmented reality projection'. Together they form a unique fingerprint.

Cite this