User Behaviour with Unguided Touchless Gestural Interfaces

Abstract

Touchless gestural interfaces provide users with the ability to use body movements and language gestures as a modality of input. Unlike other input modes, gestures must be interpreted by a recognition system, providing users with a degree of flexibility in how they are performed. Such interfaces are often suggested as a 'natural interface' alternative, allowing the utilisation of pre-learned communication skills for interaction. However, touchless interfaces are still governed by constraints, which can be opaque to users due to the complexity of the recognition system involved.

This thesis explores the intuitive behaviour users exhibit when using touchless gestural interfaces. It describes four separate studies where users specified the constraints of the touchless interfaces, to allow the system to respond to their intuitive method of gesturing. The studies provided users with a series of tasks they performed with either deictic (pointing) gestures or dynamic free-hand gestures. I recorded variations between user performances and classified individual gestures to determine if similarities existed between the way users approached such systems.

A considerable degree of variation was observed between the way users approached defining their gestures initially. Individual actions often had many possible definitions, with no significant inclination towards a common definition from the tested sample. Over time, participants typically did not make significant changes to their gesture performances, but when they did it was in response to recognition errors, fatigue, or curiosity and enjoyment. The studies also found that certain elements of the user performance could be manipulated by changing factors in the environment, such as providing a differently shaped device for performing deictic gestures.

The findings suggest that there is no pattern of common behaviour that users exhibit when trying to use a touchless interface. Additionally, there are many different potential interpretations of how such systems work, and this interpretation can significantly impact the way users perform gestures. Guiding this interpretation could be a powerful tool to interaction designers to produce intuitive behaviours conducive to a positive user experience.