Developing a situated virtual reality simulation for telerobotic control and training

Abstract

In this paper, we present the development of a situated virtual reality simulation for control and training in a telerobotic mining setting. The original research scenario is derived from a real-world rock breaking task in mining teleoperation. With the intention of having better situational awareness and user control model for this application, we simulate the entire setting in a 3D Virtual Reality (VR) environment. Therefore, users are able to obtain more information (e.g. depth information) and feedback from the remote environment in this simulation than only working with real video streams from the remote camera(s). In addition, the concept of natural interaction has been applied in building more intuitive user control interfaces than conventional manual modes. Both human eye gaze and head movements have been used to develop natural and interactive viewpoint control models for users to complete the teleoperation task. By using such a 3D simulation, training in the complex teletobotic control process can be effectively carried out with the capability of changing visual and control conditions easily. A user study has also been conducted as the preliminary evaluation of the simulation. Encouraging feedback has been provided by the experimental participants regarding task learning, which suggests the effectiveness of using the simulation.