Due to the need to interact with virtual objects, the hand-object interaction has become an important element in mixed reality (MR) applications. In this paper, we propose a novel approach to handle the occlusion of augmented 3D object manipulation with hands by exploiting the nature of hand poses combined with tracking-based and model-based methods, to achieve a complete mixed reality experience without necessities of heavy computations, complex manual segmentation processes or wearing special gloves. The experimental results show a frame rate faster than real-time and a great accuracy of rendered virtual appearances, and a user study verifies a more immersive experience compared to past approaches. We believe that the proposed method can improve a wide range of mixed reality applications that involve hand-object interactions.
|Title of host publication||Proceedings of the 2018 ACM Symposium on Virtual Reality Software and Technology|
|Publication status||Published - 28 Nov 2018|
|Event||24th ACM Symposium on Virtual Reality Software and Technology - Waseda University, Tokyo, Japan|
Duration: 28 Nov 2018 → 1 Dec 2018
|Conference||24th ACM Symposium on Virtual Reality Software and Technology|
|Abbreviated title||VRST 2018|
|Period||28/11/18 → 1/12/18|