Embodiment or Manipulation? Understanding Users' Strategies for Free-Hand Character Control

Abstract

Controlling a virtual character with your free hands is a useful task for many 3D applications such as games, computer puppetry, or emerging virtual reality applications. So far, only specialist controls have been established in the animation industry. Yet little is known about novices mental models for character control, a key to designing widely usable natural and expressive interfaces. To this end we conducted a gesture elicitation study with twelve participants performing mid-air gestures for thirteen given character motions. The mental models observed fall into two distinct categories: 1) external manipulation of an imagined physical puppet and 2) the gesturing hands embodying the motion of the virtual body part being "controlled". The employed mental model determined hand posture and the mental transformation from gesture to character motion. We present and discuss a gesture set that can inform virtual puppetry interfaces for various application domains.