Interactive image driven sound

Abstract

This paper describes the work in progress of an interactive audiovisual synthesis application that explores the relationship between user interaction, image sequences and sound in the context of interactive digital arts and media productions. The application generates aesthetically pleasing audiovisual output in interactive and real-time. It is based on an analysis of visual changes in the image sequences which are mapped to musical scales. The application is intended to become an integrated and modular toolkit that supports creative professionals in exploring different combinations of image and sound representations, for instance, in the context of mood representations, artistic ideation and content creation processes. The current implementation marks an initial step into this direction.