Identifying User Interface Patterns from Pertinent Multimodal Interaction Use Cases.

Abstract

The context of this work is usability engineering for multimodal interaction. In contrast to other work that concentrates on prototyping toolkits or abstract guidelines, this research focuses on user interface patterns for multimodal interaction. The topics of this work are not implementation centric patterns but rather user-task-near interface patterns. On the one hand higher level patterns that are based on the general principles of the multimodal design space (patterns of multimodal combination and multimodal adaptation) are described, as well as more concrete use case specific patterns on the other hand. At first pattern candidates are derived from knowledge about how multimodality can enhance usability. At the same time literature is mined for real solutions as patterns are only valid as long as they describe proven solutions from the real world. Along with this, relationships between patterns are depicted in the context of these interaction use cases.