Music Technology for Enhancing Creative Expression and Education

July 30, 2008
Youngmoo Kim | Drexel University

Music is perhaps the richest medium for the expression of human emotions, but the computational analysis and synthesis of emotion through music is still in its infancy. This presentation will describe several research projects that explore the relationship between musical expression and emotion under the common vision of making music more interactive and accessible for both musicians and non-musicians.

The problem of detecting and labeling emotions within music is not only difficult to solve computationally, but is often without well- defined answers. The lack of clear “ground truth” makes it difficult to train systems that rely on quantified labels for supervised machine learning. Recently, there have been many initiatives to use online collaborative games for the collection of label data, and several such games have been proposed to collect labels spanning an excerpt of music. We have developed a new game, MoodSwings, that differs in that it records dynamic (per-second) labels of players’ mood ratings of music, in keeping with the unique time-varying quality of musical mood. Furthermore, we believe these types of activities can be designed to simultaneously educate users, particularly K-12 students, about aspects of music information and acoustics. Our lab has created interactive activities illustrating aspects of two different sound and acoustics concepts: musical instrument timbre and the “cocktail party problem” (sound source isolation within mixtures) that also provide an instrument for collecting perceptual data related to these problems with a range of parameter variation that is difficult to achieve for large subject populations using traditional psychoacoustic evaluation.

At the same time, interest in music “performance” games (e.g., Guitar
Hero) is growing while music education is disappearing from public school curricula. While it is unclear whether such games have any impact on music skill development, the current design of these games leaves little room for musical expression or creativity during gameplay. Our lab is pursuing research in new musical interfaces (multi-touch, gesture-based, and robotic platforms) and activities that allow us to rapidly prototype and explore the expressive capabilities of interfaces with different forms of feedback. Our collaborations with middle- and high-school programs allows us to evaluate these new interfaces using students with and without prior musical training.

Speaker Details

Youngmoo Kim is an Assistant Professor of Electrical and Computer Engineering Department at Drexel University. He received his Ph.D. from the MIT Media Lab in 2003 and also holds Masters degrees in Electrical Engineering and Music (Vocal Performance Practice) from Stanford University. His dissertation explored analysis and synthesis of the singing voice, investigating acoustic features relating to voice identity. While at MIT, he was also an active member of the MPEG standards committee, contributing to the MPEG-4 and MPEG-7 audio standards. During his tenure in Boston, he was also a member of the Tanglewood Festival Chorus, the chorus of the Boston Symphony Orchestra. Prior to his current position at Drexel, Youngmoo was Senior Research Scientist at Nellymoser, Inc., a company focused on audio technologies for console video games (including the voice codec for Xbox Live) and mobile entertainment. Youngmoo’s research interests include audio signal processing, machine listening, and music information retrieval. His lab, the Music and Entertainment Technology (MET) lab currently supports 4 graduate students and several undergraduate researchers, and his research is supported by the National Science Foundation. He recently chaired the Robotics & Creativity Workshop at AAAI 2008, and he is co-chair of the 2008 International Conference on Music Information Retrieval (ISMIR) to be held in September at Drexel.