{"id":326648,"date":"2016-11-23T14:01:47","date_gmt":"2016-11-23T22:01:47","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-project&p=326648"},"modified":"2021-05-09T12:03:48","modified_gmt":"2021-05-09T19:03:48","slug":"user-specific-training-vocal-melody-transcription","status":"publish","type":"msr-project","link":"https:\/\/www.microsoft.com\/en-us\/research\/project\/user-specific-training-vocal-melody-transcription\/","title":{"rendered":"User-Specific Training for Vocal Melody Transcription"},"content":{"rendered":"

Overview<\/b>
\nThis page contains supplementary material for our AAAI 2010 paper: \u201cUser-Specific Learning for Recognizing a Singer\u2019s Intended Pitch\u201d. The full citation for our paper follows, along with a link to the paper itself:<\/p>\n

Guillory A, Basu S, and Morris D. User-Specific Learning for Recognizing a Singer\u2019s Intended Pitch<\/a>. Proceedings of AAAI 2010, July 2010.<\/p>\n

For more information about this work, contact Dan Morris (dan@microsoft.com<\/a>) and Sumit Basu (sumitb@microsoft.com<\/a>).<\/p>\n

Abstract<\/b>
\nWe consider the problem of automatic vocal melody transcription: translating an audio recording of a sung melody into a musical score. While previous work has focused on finding the closest notes to the singer\u2019s tracked pitch, we instead seek to recover the melody the singer intended to sing. Often, the melody a singer intended to sing differs from what they actually sang; our hypothesis is that this occurs in a singer-specific way. For example, a given singer may often be flat in certain parts of her range, or another may have difficulty with certain intervals. We thus pursue methods for singer-specific training which use learning to combine different methods for pitch prediction. In our experiments with human subjects, we show that via a short training procedure we can learn a singer-specific pitch predictor and significantly improve transcription of intended pitch over other methods. For an average user, our method gives a 20 to 30 percent reduction in pitch classification errors with respect to a baseline method comparable to commercial voice transcription tools. For some users, we achieve even more dramatic reductions. Our best results come from a combination of singer-specificlearning with non-singer-specific feature selection. We are also making our experimental data available to allow others to replicate or extend our results, and we discuss the implications of our work for training more general control signals.<\/p>\n

Supplementary Material<\/b>
\nThe primary purpose of this page is to host the data used in our experiments, which consist of:<\/p>\n

    \n
  1. A series of MIDI tracks, each a short melody or scale, several measures long<\/li>\n
  2. Recordings of 22 participants singing those melodies along with a drum beat, time-sync’d to the original MIDI tracks<\/li>\n<\/ol>\n

    We hope these recordings can serve as the beginning of a larger data repository, and as a benchmark data set for user-specific training or vocal melody transcription for environments with fixed tempos (an important feature of these recordings is that they were created by asking users to sing along with a drum beat).<\/p>\n

    Our complete experimental procedure is described in detail in our paper, and the instructions displayed to participants will be included at the end of this page. We note that only 22 recordings are included here, which is smaller than the total number collected: not all participants consented to having their recordings publicly released. However, the data set posted on this page is not systematically biased and is appropriate for testing alternate methods and understanding our experiments.<\/p>\n

    Our data archive can be downloaded as a single zipfile:<\/p>\n

    public_pitch_data.zip (opens in new tab)<\/span><\/a> (560MB)<\/p>\n

    The archive contains two directories:<\/p>\n

    \n

    input_data<\/i><\/b><\/p>\n

    This directory contains the ground truth files in both MIDI and audio format. Each file is numbered and named to correspond to the recordings for each participant (described below). In some cases files are labeled as \u201cmale\u201d or \u201cfemale\u201d; these represent the same melody in slightly different ranges to allow for reasonable reproduction by participants of both genders. Each example is included as a MIDI sequence (.mid), an \u201cexample\u201d sequence (.wma) (this is what participants heard before they were asked to sing back each example), and an \u201caccompaniment\u201d sequence (this is what participants heard while they were singing back each example: just a drum beat, an initial cue to set the key, and a count-in voiceover.<\/p>\n

    For example, melody 2 is represented in this directory as six files (all of which are live links on this page, as examples):<\/p>\n

    02TwinkleFemale.accompaniment.wma (opens in new tab)<\/span><\/a>
    \n    02TwinkleFemale.example.wma (opens in new tab)<\/span><\/a>
    \n    02TwinkleFemale.mid (opens in new tab)<\/span><\/a>
    \n    02TwinkleMale.accompaniment.wma (opens in new tab)<\/span><\/a>
    \n    02TwinkleMale.example.wma (opens in new tab)<\/span><\/a>
    \n    02TwinkleMale.mid (opens in new tab)<\/span><\/a><\/p>\n

    public<\/i><\/b><\/p>\n

    This directory contains recordings from 22 partipcants singing along with each of our 21 melodies. For example, the directory called \u201cP004\u201d contains participant 4\u2019s recordings of all 21 melodies, so files are named:<\/p>\n

    00Easy1.training.wav
    \n01Easy2.training.wav
    \n02TwinkleMale.training.wav
    \n03Diddle2Male.training.wav
    \n…
    \n21OdeToJoyMale.training.wav<\/p>\n<\/div>\n

    The remainder of this page contains a brief video example of our data collection application, along with the instructions provided to participants, to demonstrate the process used for data collection. Please contact Dan Morris (dan@microsoft.com<\/a>) and Sumit Basu (sumitb@microsoft.com<\/a>) if you have questions about our procedure, are interested in implementing a competing method, or are interested in adding to our data repository!<\/p>\n

    Data Collection App Video Clip<\/strong><\/p>\n