Program dates: January 2022-September 2022
The INTERSPEECH 2022 (opens in new tab) Audio Deep Packet Loss Concealment Challenge is intended to stimulate research in the area of Audio Packet Loss Concealment (PLC).
The training and validation dataset has now been released: microsoft/PLC-Challenge (opens in new tab) – additional info can be found in the draft challenge paper (opens in new tab)
PLC is an important part of audio telecommunications technology and codec development, and methods for performing PLC using machine learning approaches are now becoming viable for practical use. Packet loss, either by missing packets or high packet jitter, is one of the top reasons for speech quality degradation in Voice over IP calls.
While there have been some groups publishing in this area, a lack of common datasets and evaluation procedures complicates the comparison of proposed methods and the establishment of clear baselines. With this challenge, we propose to address this situation: We will open source a dataset based on real-world (as opposed to the common synthetic) packet loss traces and bring the community together to, for the first time, compare approaches in this field on a unified test set.
As the gold standard for audio quality evaluation is human evaluator ratings, we will evaluate submissions using a crowd-source ITU-T P.808 CCR approach. The top three approaches which achieve the highest average Mean Opinion Score on the blind set will be declared the winners of the INTERSPEECH 2022 Audio Deep Packet Loss Concealment Challenge. As an additional metric, to ensure that approaches are not degrading intelligibility, we will use the speech recognition rate, calculated using the Microsoft Cognitive Services Speech Recognition Service.
To help participants during the challenge, we will provide participants with access to our prototype “PLC-MOS” neural network model that provides estimates of human ratings of audio files with healed packet losses.
Registration procedure
To register for the challenge, please register on the Challenge CMT (opens in new tab) site where they can then submit the enhanced clips. To get access to the PLC-MOS service, please fill out the registration form (opens in new tab). All communication will be done via CMT for the challenge, including challenge updates and results.
Submission instructions
Please use the Microsoft Conference Management Toolkit (opens in new tab) for submitting the results. After logging in, complete the following steps to submit the results:
- Choose “Create new submission” in the Author Console.
- Enter title (this can be changed later), abstract and co-authors, and upload a lastname.txt file (can be empty or contain additional information regarding the submission).
- Compress the enhanced results files to a single lastname.zip file, retaining the same folder and file names as the blind test set (max file size: 350 MB).
- After creating the submission, return to the “Author Console” (by clicking on “Submissions” at the top of the page) and upload the lastname.zip file via “Upload Supplementary Material”.
Submission deadline: March 8, 2022, 11:59 PM (anywhere on earth)
Contact us: For questions, please contact plc_challenge@microsoft.com