Noise suppression has become more important than ever before due to the increasing use of voice interfaces for various applications. Given\u202fthe\u202fmillions of internet-connected devices being employed for audio\/video calls, noise suppression is expected to be effective for all noise types chosen from daily-life scenarios.\u202fThe\u202fIEEE ICASSP 2022\u202fGrand\u202fChallenge is the 4th DNS\u202fchallenge intended to promote industry-academia collaboration on research in real-time noise suppression aimed to maximize the subjective (perceptual) quality of enhanced speech.\u202fThis challenge will extend DNS efforts to\u202ffull band\u202fspeech with a special focus on personalized denoising. In\u202fthe era of hybrid work, personalized denoising is very important to suppress neighboring speaker and\/or background noises.\u202fRecently, DNS research\u202fhas been\u202fmoving fast, and researchers now have state-of-the-art advancements\u202fin deep neural networks (DNNs);\u202fcurrently, deep noise suppression methods leverage the convolutional, recurrent, or hybrid neural network for estimating the enhanced speech from noisy recordings.<\/p>\n\n\n\n
Previous editions of\u202fthe\u202fDNS Challenge provided researchers with a massive training dataset and real test set along with\u202fa\u202fP.808\/P.835\u202ftest framework\u202ffor subjective evaluation\u202fof enhanced speech. In\u202fthe\u202fcurrent\u202fchallenge, we improved the training dataset by cleaning it further and added some more data to\u202fcapture\u202frelevant\u202fDNS scenarios.\u202fWe\u202fcollected\u202fa new test set for\u202ffull band\u202fspeech ensuring high energy content in\u202fhigher\u202ffrequency bands\u202fto eliminate bandlimited clips from some devices. We\u202fincluded new noise types in the test set\u202fcovering contemporary\u202fscenarios and device variety, especially mobile scenarios. Our training data synthesizer script is flexible to allow the exclusion of any subset or addition of new data\u202fby the challenge\u202fparticipants.\u202fThis provides an opportunity for leveraging challenge data along with other corpora for improving DNS performance. Our test set consists of real-world test clips\u202frecorded by crowd-sourced workers and\/or Microsoft employees. We have two dev-test sets for real-time denoising and personalized real-time denoising. Similarly, we have two blind test sets, one for each challenge track.<\/p>\n\n\n\n
Challenge paper can be found ICASSP_2022_4th_Deep_Noise_Suppression_Challenge<\/a><\/p>\n\n\n\n The tracks in this challenge are:<\/p>\n\n\n\n Track 1:\u202fReal-Time\u202fnon-personalized DNS\u202ffor\u202ffull band\u202fspeech\u202f<\/b><\/p>\n\n\n\n Track 2: Real-Time\u202fPersonalized DNS\u202ffor\u202ffull band\u202fspeech<\/b><\/p>\n\n\n\n Participants are forbidden from using the blind test set to retrain or tweak their models. Participants must submit results only if they intend to submit a paper to <\/b>ICASSP 2022 Deep Noise Suppression Challenge<\/b>. Failing to adhere to these rules will lead to disqualification from the challenge.<\/b><\/p>\n\n\n\n This challenge\u202fadopts the\u202fITU-T P.835 subjective\u202ftest framework\u202fto measure speech quality (SIG), background noise quality (BAK), and overall audio quality (OVRL). We are also releasing DNSMOS\u202fP.835 (opens in new tab)<\/span><\/a>, which is a machine learning\u202fbased\u202fmodel for predicting SIG, BAK, OVRL. Participants can use DNSMOS P.835 to evaluate their intermediate models.\u202fIn this challenge, we introduced\u202fWord Accuracy (WAcc)\u202fas\u202fan additional metric to compare the performance of DNS models. Challenge winners will be decided based on OVRL and WAcc as follows:<\/p>\n\n\n\n WAcc\u202fwill be obtained using\u202fMicrosoft\u202fAzure Speech Recognition API. This challenge metric gives an equal weighting between subjective quality and speech recognition performance. The dev-test set and\u202fDNSMOS\u202fP.835 are provided to participants to accelerate model development.\u202fA script to evaluate WAcc is also provided.\u202fWe neither use the dev-test set nor\u202fDNSMOS\u202fP.835 for deciding final winners.\u202fDNSMOS\u202fP.835 has a high correlation with human perception and hence can serve as a robust measure of audio quality. Challenge winner will be decided based on M<\/em> computed on enhanced clips from blind\u202ftest set.<\/p>\n\n\n\n Contact us: <\/strong>If you have questions about this program, email us at dns_challenge@microsoft.com<\/a>.<\/p>\n\n\n\n\n\n SPONSOR<\/p>\n\n\n\n These Official Rules (\u201cRules\u201d) govern the operation of the ICASSP 2022 Deep Noise Suppression Challenge Event Contest (\u201cContest\u201d). Microsoft Corporation, One Microsoft Way, Redmond, WA, 98052, USA, is the Contest sponsor (\u201cSponsor\u201d).<\/p>\n\n\n\n DEFINITIONS<\/p>\n\n\n\n In these Rules, \u201cMicrosoft\u201d, \u201cwe\u201d, \u201cour\u201d, and \u201cus\u201d, refer to Sponsor, and \u201cyou\u201d and \u201cyourself\u201d refers to a Contest participant or the parent\/legal guardian of any Contest entrant who has not reached the age of majority to contractually obligate themselves in their legal place of residence. \u201cEvent\u201d refers to the ICASSP 2022 Deep Noise Suppression event held in Singapore (the \u201cEvent\u201d). By entering you (your parent\/legal guardian if you are not the age of majority in your legal place of residence) agree to be bound by these Rules.<\/p>\n\n\n\n ENTRY PERIOD<\/p>\n\n\n\n The Contest will operate from December 1, 2021 to January 20, 2022 (\u201cEntry Period\u201d). The Entry Period is divided into several periods as described in Section 5 How to Enter.<\/p>\n\n\n\n ELIGIBILITY<\/p>\n\n\n\n Open to any registered Event attendee 18 years of age or older. If you are 18 years of age or older but have not reached the age of majority in your legal place of residence, then you must have the consent of a parent\/legal guardian. Employees and directors of Microsoft Corporation and its subsidiaries, affiliates, advertising agencies, and Contest Parties are not eligible, nor are persons involved in the execution or administration of this promotion, or the family members of each above (parents, children, siblings, spouse\/domestic partners, or individuals residing in the same household). Void in Cuba, Iran, North Korea, Sudan, Syria, Region of Crimea, and where prohibited. For business\/tradeshow events: If you are attending the Event in your capacity as an employee, it is your sole responsibility to comply with your employer\u2019s gift policies. Microsoft will not be a party to any disputes or actions related to this matter. PLEASE NOTE: If you are a public sector employee (government and education), all prize awards will be awarded directly to your public sector organization and subject to receipt of a gift letter signed by your agency\/institution\u2019s ethics officer, attorney, or designated executive\/officer responsible for your organization\u2019s gifts\/ethics policy. Microsoft seeks to ensure that by offering items of value at no charge in promotional settings it does not create any violation of the letter or spirit of the entrant\u2019s applicable gifts and ethics rules.<\/p>\n\n\n\n HOW TO ENTER<\/p>\n\n\n\n The Contest Objective is to promote collaborative research in real-time single-channel Speech Enhancement aimed to maximize the subjective (perceptual) quality of the enhanced speech. Prizes will be awarded based on the speech quality of deep noise suppression models using the online subjective evaluation framework ITU-T P.835. Only methods described in accepted ICASSP 2022 Deep Noise Suppression Challenge papers will be eligible for the contest. You may participate as an individual or a team. If forming a team, you must designate a \u201cTeam Captain\u201d who will submit all entry materials on behalf of the team. Once you register as part of a Team, you cannot change Teams or alter your current team (either by adding or removing members) after the submission of your Entry. Limit one Entry per person and per team. You may not compete on multiple teams and you may not enter individually and on a team. We are not responsible for Entries that we do not receive for any reason, or for Entries that we receive but are not decipherable or not functional for any reason. Each Team is solely responsible for its own cooperation and teamwork. In no event will Microsoft officiate in any dispute regarding the conduct or cooperation of any Team or its members. The Contest will operate as follows:<\/p>\n\n\n\n Registration \/ Development Period: December 1, 2021 \u2013 January 20, 2022 Create a submission by registering at Conference Management Toolkit \u2013 DNS Challenge 2022 (opens in new tab)<\/span><\/a> and fill in all your details. You will be using this tool for final submission and to receive any email announcements from organizers.<\/p>\n\n\n\n Then 1) develop a speech enhancement model that best meets the Contest Objective as described in the base paper and 2) computational complexity of the model in terms of the number of parameters and the time it takes to infer a frame on a particular CPU (preferably Intel Core i5 quad core machine clocked at 2.4 GHz). To develop your model, use any publicly available clean speech and noise datasets, including the contest datasets provided for training and developing models. You may augment your datasets with the contest dataset. You may mix clean speech and noise in any way that improves the performance of your model.<\/p>\n\n\n\n The final evaluation will be conducted on a blind test set that is similar to the open-sourced development stage test set. You may use scripts for a baseline noise suppressor that was recently published here. Testing \/ Entry Period: December 1, 2021 \u2013 January 20, 2022. On January 15, the blind test dataset will be released. You will have until 11:59 PM PDT on January 20, 2022 to test your model against this dataset and create a set of enhanced clips to submit for judging (your \u201cEntry\u201d) via Conference Management Tool.<\/p>\n\n\n\n You may not use the blind test set to retrain or tweak your model. To submit your entry, submit your processed clips via conference management tool. Each Entry will fall in one of two tracks based on whether it is personalized or non-personalized DNS. You must satisfy all the requirements of each track in terms of algorithmic latency. You must also specify the Number of operations per second in your paper submission. ICASSP 2022 Paper Submission and Judging Period: February 3,2022 11:59 PM PDT \u2013 February 10, 2022. Your Entry must be described in a paper accepted by ICASSP 2022 Deep Noise Suppression Challenge. To submit a paper, use the Conference Management Toolkit \u2013 DNS Challenge 2022 (opens in new tab)<\/span><\/a>. The entry limit is one per person during the Entry Period. Any attempt by any you to obtain more than the stated number of entries by using multiple\/different accounts, identities, registrations, logins, or any other methods will void your entries and you may be disqualified. Use of any automated system to participate is prohibited.<\/p>\n\n\n\n We are not responsible for excess, lost, late, or incomplete entries. If disputed, entries will be deemed submitted by the \u201cauthorized account holder\u201d of the email address, social media account, or other method used to enter. The \u201cauthorized account holder\u201d is the natural person assigned to an email address by internet or online service provider, or other organization responsible for assigning email addresses.<\/p>\n\n\n\n PAPER FORMAT<\/p>\n\n\n\n The Challenge papers are 4 pages + 1 reference page and use the format defined here: https:\/\/2022.ieeeicassp.org\/papers\/paper_kit.php (opens in new tab)<\/span><\/a><\/p>\n\n\n\n ELIGIBLE ENTRY<\/p>\n\n\n\n To be eligible, an entry must meet the following content\/technical requirements:<\/p>\n\n\n\n USE OF ENTRIES<\/p>\n\n\n\n We are not claiming ownership rights to your Submission. However, by submitting an entry, you grant us an irrevocable, royalty-free, worldwide right and license to use, review, assess, test, and otherwise analyze your entry and all its content in connection with this Contest and use your entry in any media whatsoever now known or later invented for any non-commercial or commercial purpose, including, but not limited to, the marketing, sale or promotion of Microsoft products or services, without further permission from you. You will not receive any compensation or credit for use of your entry, other than what is described in these Official Rules.<\/p>\n\n\n\n By entering you acknowledge that we may have developed or commissioned materials similar or identical to your entry and you waive any claims resulting from any similarities to your entry. Further, you understand that we will not restrict work assignments of representatives who have had access to your entry and you agree that the use of information in our representatives\u2019 unaided memories in the development or deployment of our products or services does not create liability for us under this agreement or copyright or trade secret law.<\/p>\n\n\n\n Your entry may be posted on a public website. We are not responsible for any unauthorized use of your entry by visitors to this website. We are not obligated to use your entry for any purpose, even if it has been selected as a winning entry.<\/p>\n\n\n\n WINNER SELECTION AND NOTIFICATION<\/p>\n\n\n\n Pending confirmation of eligibility, potential prize winners will be selected by Microsoft or their Agent or a qualified judging panel from among all eligible entries received based on the following judging criteria: 99% \u2013 The subjective speech quality evaluated on the blind test set using ITU-T P.835 framework. We will use the submitted clips with no alteration to conduct ITU-T P.835 subjective evaluation. We will use Word Accuracy (WAcc) as an additional metric to compare the performance of DNS models. WAcc will be obtained using Microsoft Azure Speech Recognition API. Challenge winners will be decided based on Overall MOS (OVRL) rating from the ITU-T P.835 subjective evaluation results and WAcc as follows: M=((OVLR-1)\/4+WAcc)\/2.<\/p>\n\n\n\n This challenge metric gives an equal weighting between subjective quality and speech recognition performance. Among the submitted proposals, if the difference between overall evaluation metric M between the models is not statistically significant, the model with a lower number of operations per second be given a higher ranking. 1% \u2013 The Entry was described in an accepted ICASSP 2022 Deep Noise Suppression Challenge paper. Winners will be selected within 7 days following the event. Winners will be notified within 7 days following the Event.<\/p>\n\n\n\n In the event of a tie between any eligible entries, an additional judge will break the tie based on the judging criteria described above. The decisions of the judges are final and binding. If we do not receive enough entries meeting the entry requirements, we may, at our discretion, select fewer winners. If public vote determines winners, it is prohibited for any person to obtain votes by any fraudulent or inappropriate means, including offering prizes or other inducements in exchange for votes, automated programs, or fraudulent IDs. Microsoft will void any questionable votes.<\/p>\n\n\n\n ODDS<\/p>\n\n\n\n The odds of winning are based on the number and quality of eligible entries received.<\/p>\n\n\n\n GENERAL CONDITIONS AND RELEASE OF LIABILITY<\/p>\n\n\n\n To the extent allowed by law, by entering you agree to release and hold harmless Microsoft and its respective parents, partners, subsidiaries, affiliates, employees, and agents from any and all liability or any injury, loss, or damage of any kind arising in connection with this Contest or any prize won.<\/p>\n\n\n\n All local laws apply. The decisions of Microsoft are final and binding.<\/p>\n\n\n\n We reserve the right to cancel, change, or suspend this Contest for any reason, including cheating, technology failure, catastrophe, war, or any other unforeseen or unexpected event that affects the integrity of this Contest, whether human or mechanical. If the integrity of the Contest cannot be restored, we may select winners from among all eligible entries received before we had to cancel, change or suspend the Contest.<\/p>\n\n\n\n If you attempt or we have strong reason to believe that you have compromised the integrity or the legitimate operation of this Contest by cheating, hacking, creating a bot or other automated program, or by committing fraud in any way, we may seek damages from you to the full extent of the law and you may be banned from participation in future Microsoft promotions.<\/p>\n\n\n\n GOVERNING LAW<\/p>\n\n\n\n This Contest will be governed by the laws of the State of Washington, and you consent to the exclusive jurisdiction and venue of the courts of the State of Washington for any disputes arising out of this Contest.<\/p>\n\n\n\n PRIVACY<\/p>\n\n\n\n At Microsoft, we are committed to protecting your privacy. Microsoft uses the information you provide on this form to notify you of important information about our products, upgrades and enhancements, and to send you information about other Microsoft products and services. Microsoft will not share the information you provide with third parties without your permission except where necessary to complete the services or transactions you have requested, or as required by law. Microsoft is committed to protecting the security of your personal information. We use a variety of security technologies and procedures to help protect your personal information from unauthorized access, use, or disclosure. Your personal information is never shared outside the company without your permission, except under conditions explained above.<\/p>\n\n\n\n If you believe that Microsoft has not adhered to this statement, please contact Microsoft by sending an email to\u202fprivrc@microsoft.com<\/a>\u202for postal mail to Microsoft Privacy Response Center, Microsoft Corporation, One Microsoft Way, Redmond, WA.<\/p>\n\n\n\n\n\n Time zone for below dates is Anywhere on Earth<\/em> (AoE)<\/p>\n\n\n\n Results: P.835 subjective evaluation for Track 1 non-personalized DNS. <\/strong>DMOS is difference of MOS between enhanced speech and noisy speech.<\/strong><\/p>\n\n\n\n ANOVA results for Track-1:<\/strong><\/p>\n\n\n\n\n
\n
Evaluation criteria and methodology<\/h3>\n\n\n\n
Registration procedure<\/h3>\n\n\n\n
\n
Official rules<\/h2>\n\n\n\n
To register, please send an email to dns_challenge@microsoft.com<\/a> stating that you are interested to participate in the challenge. Please include the following details in your email:<\/p>\n\n\n\n\n
\n
Program timeline<\/h2>\n\n\n\n
\n
Organizers<\/h2>\n\n\n\n
\n
Related links<\/h2>\n\n\n\n
\n
Other challenges<\/h3>\n\n\n\n
\n
![\"p835_final_v7\"](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/11\/p835_final_v7.png\")
<\/a><\/figure>\n\n\n\n
![\"p835_pdns_final_v7.png\"](\"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2021\/11\/p835_pdns_final_v7.png\")
<\/a><\/figure>\n\n\n\n