VocalSketch: Vocally Imitating Audio Concepts

Cartwright, M., Pardo, B. VocalSketch: Vocally Imitating Audio Concepts. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), 2015. Honorable Mention

Abstract

A natural way of communicating an audio concept is to imitate it with one’s voice. This creates an approximation of the imagined sound (e.g. a particular owl’s hoot), much like how a visual sketch approximates a visual concept (e.g a drawing of the owl). If a machine could understand vocal imitations, users could communicate with software in this natural way, enabling new interactions (e.g. programming a music synthesizer by imitating the desired sound with one’s voice). In this work, we collect thousands of crowd-sourced vocal imitations of a large set of diverse sounds, along with data on the crowd’s ability to correctly label these vocal imitations. The resulting data set will help the research community understand which audio concepts can be effectively communicated with this approach. We have released the data set so the community can study the related issues and build systems that leverage vocal imitation as an interaction modality.

Resources