AI - Driven Hand Reconstruction for Immersive Teleoperation in Construction Industry: Bridging Physical and Digital Realities with RGB- D Reconstruction

Document Type

Conference Proceeding

Publication Date

1-1-2025

Abstract

The construction industry is increasingly adopting robotic systems, yet intuitive control for complex manipulation remains a significant challenge. This paper introduces a new paradigm for human-robot teleoperation in construction, centered on high-fidelity, real-time reconstruction of the operator's hand. Our method adapts recent advances in analysis-by-synthesis, employing a differentiable renderer to jointly optimize hand shape, pose, appearance, and ambient lighting from a single RGB-D sensor. By integrating a photometric loss with an articulated point-to-plane ICP energy, the framework robustly tracks the hand even during complex interactions. We validate our system's suitability as a high-quality control source by demonstrating state-of-the-art performance on challenging hand-object interaction datasets, achieving a Mean Per-Joint Position Error (MPJPE) of 8.6 mm on DexYCB and 5.4 mm on HO-3D. These results confirm that our method produces a control signal of sufficient fidelity and robustness for the nuanced demands of tool manipulation, representing a foundational step toward safer and more efficient human-robot collaboration in construction.

Publication Source (Journal or Book title)

2025 IEEE 4th International Conference on Intelligent Reality Icir 2025

This document is currently not available here.

Share

COinS