A Multimodal Assistive System for Helping Visually Impaired in Social Interactions

Abstract

Access to non-verbal cues in social interactions is vital for people with visual impairment. It has been shown that non-verbal cues such as eye contact, number of people, their names and positions are helpful for individuals who are blind. In this paper, we present a real time multi-modal system that provides such non-verbal cues via audio and haptic interfaces. We assess the usefulness of the proposed system both quantitatively and qualitatively by gathering feedback with a focus group of visually impaired participants in a typical social interaction setting. The paper provides important insight about developing such technology for this significant part of society.