TY - GEN
T1 - Video-annotated augmented reality assembly tutorials
AU - Yamaguchi, Masahiro
AU - Mori, Shohei
AU - Mohr, Peter
AU - Tatzgern, Markus
AU - Stanescu, Ana
AU - Saito, Hideo
AU - Kalkofen, Denis
N1 - Funding Information:
This work was enabled by the Competence Center VRVis, the FFG (grant 859208 - Matahari) and the Austrian Science Fund grant P30694, and partly by JST CREST under Grant JPMJCR1683, Japan. VRVis is funded by BMVIT, BMWFW, Styria, SFG and Vienna Business Agency in the scope of COMET, Competence Centers for Excellent Technologies (854174), which is managed by FFG.
Publisher Copyright:
© 2020 ACM.
PY - 2020/10/20
Y1 - 2020/10/20
N2 - We present a system for generating and visualizing interactive 3D Augmented Reality tutorials based on 2D video input, which allows viewpoint control at runtime. Inspired by assembly planning, we analyze the input video using a 3D CAD model of the object to determine an assembly graph that encodes blocking relationships between parts. Using an assembly graph enables us to detect assembly steps that are otherwise difficult to extract from the video, and generally improves object detection and tracking by providing prior knowledge about movable parts. To avoid information loss, we combine the 3D animation with relevant parts of the 2D video so that we can show detailed manipulations and tool usage that cannot be easily extracted from the video. To further support user orientation, we visually align the 3D animation with the real-world object by using texture information from the input video. We developed a presentation system that uses commonly available hardware to make our results accessible for home use and demonstrate the effectiveness of our approach by comparing it to traditional video tutorials.
AB - We present a system for generating and visualizing interactive 3D Augmented Reality tutorials based on 2D video input, which allows viewpoint control at runtime. Inspired by assembly planning, we analyze the input video using a 3D CAD model of the object to determine an assembly graph that encodes blocking relationships between parts. Using an assembly graph enables us to detect assembly steps that are otherwise difficult to extract from the video, and generally improves object detection and tracking by providing prior knowledge about movable parts. To avoid information loss, we combine the 3D animation with relevant parts of the 2D video so that we can show detailed manipulations and tool usage that cannot be easily extracted from the video. To further support user orientation, we visually align the 3D animation with the real-world object by using texture information from the input video. We developed a presentation system that uses commonly available hardware to make our results accessible for home use and demonstrate the effectiveness of our approach by comparing it to traditional video tutorials.
KW - Assembly tutorial.
KW - Augmented reality
KW - Retargeting
KW - Video label
UR - http://www.scopus.com/inward/record.url?scp=85096966189&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096966189&partnerID=8YFLogxK
U2 - 10.1145/3379337.3415819
DO - 10.1145/3379337.3415819
M3 - Conference contribution
AN - SCOPUS:85096966189
T3 - UIST 2020 - Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology
SP - 1010
EP - 1022
BT - UIST 2020 - Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology
PB - Association for Computing Machinery, Inc
T2 - 33rd Annual ACM Symposium on User Interface Software and Technology, UIST 2020
Y2 - 20 October 2020 through 23 October 2020
ER -