Immersive Control Rooms for Human-Robot Teams

Abstract

Human-robot teams are increasingly being used in complex and dangerous environments. At the same time, there is growing interest in the potential of immersive technologies such as virtual, augmented, and mixed reality (VR, AR, MR) for revolutionising command and control systems. Control rooms are dynamic high-information environments, where decision-making relies on fast and accurate information and clear communication to manage and respond to time-critical situations. However, control rooms are expensive and logistically challenging to set up and maintain. Further, the presentation of information to control room operators is challenging due to the boundaries of available space and the limits of traditional 2D interfaces. VR offers new opportunities for immersive communication and collaboration in human-robot teaming (HRT) for improving situational awareness to support informed decision- making. For example, VR can enable visualisation of complex information, embodied interaction for enhanced communication, teleoperation and control of remote robots and swarms, and enhanced spatial understanding of complex environments. In this paper, we survey the existing research on immersive control rooms for human-robot teams and discuss challenges and open questions.