Detecting of and Interacting with Text in Free Space

Abstract

Computer Vision technology has seen a significant push in popularity in consumer applications, whether for face recognition in today's smartphone camera apps, or server-based object recognition that identifies products, text or contexts in images within seconds. And thanks to powerful smartphone and tablet devices, computer vision application can now perform the processing locally, providing real-time feedback to the user, about the object they are looking at just by pointing the camera lens at it without the need of a remote server. Hence, new ways of interacting with one's environment become possible in particular for people with specific access needs, like vision-impaired users. Blindsight's text detection algorithm showcases a way on how to detect and speak out text in virtually real time using a smartphone. In this demo, users will be able to try out text detection and immerse into a non-visual user-experience showing the state of the art for text detection and assistive technology for the visually impaired. Based on the user's experiences, this demo shall also discuss challenges and new ways to interact with text in free space. How can technology guide a user to find the desired text, how can technology filter the right information or group the information based on context, conventions or user preferences? Providing ideas and answers for these question will not only enrich the advancements of assistive technology, it will also inspire new application in a world where visual experiences are more and more supplemented by tactile, audible and vocal user interfaces.