Add-on Occlusion: Turning Off-the-Shelf Optical See-through Head-mounted Displays Occlusion-capable
Yan Zhang1, Xiaodan Hu2, Kiyoshi Kiyokawa2, and Xubo Yang1†
1: Shanghai Jiao Tong University
2: Nara Institute of Science and Technology
†: Corresponding author
Abstract
The occlusion-capable optical see-through head-mounted display (OC-OSTHMD) is actively developed in recent years since it allows mutual occlusion between virtual objects and the physical world to be correctly presented in augmented reality (AR). However, implementing occlusion with the special type of OSTHMDs prevents the appealing feature from the wide application. In this paper, a novel approach for realizing mutual occlusion for common OSTHMDs is proposed. A wearable device with per-pixel occlusion capability is designed. OSTHMD devices are upgraded to be occlusion-capable by attaching the device before optical combiners. A prototype with HoloLens 1 is built. The virtual display with mutual occlusion is demonstrated in real-time. A color correction algorithm is proposed to mitigate the color aberration caused by the occlusion device. Potential applications, including the texture replacement of real objects and the more realistic semi-transparent objects display, are demonstrated. The proposed system is expected to realize a universal implementation of mutual occlusion in AR.
Keywords: Augmented reality, near-to-eye displays, occlusion displays, head-mounted displays, diminished reality, color blending
Video
Citation
Y. Zhang, X. Hu, K. Kiyokawa and X. Yang, “Add-on Occlusion: Turning Off-the-Shelf Optical See-through Head-mounted Displays Occlusion-capable,” in IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 5, pp. 2700-2709, May 2023, doi: 10.1109/TVCG.2023.3247064.