{"id":848614,"date":"2022-05-30T01:26:18","date_gmt":"2022-05-30T08:26:18","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/"},"modified":"2023-04-18T00:21:42","modified_gmt":"2023-04-18T07:21:42","slug":"a-few-thousand-translations-go-a-long-way-leveraging-pre-trained-models-for-african-news-translation","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/a-few-thousand-translations-go-a-long-way-leveraging-pre-trained-models-for-african-news-translation\/","title":{"rendered":"A Few Thousand Translations Go A Long Way! Leveraging Pre-trained Models for African News Translation"},"content":{"rendered":"<p><strong>Abstract:<\/strong> Recent advances in the pre-training for language models leverage large-scale datasets to create multilingual models. However, low-resource languages are mostly left out in these datasets. This is primarily because many widely spoken languages that are not well represented on the web and therefore excluded from the large-scale crawls for datasets. Furthermore, downstream users of these models are restricted to the selection of languages originally chosen for pre-training. This work investigates how to optimally leverage existing pre-trained models to create low-resource translation systems for 16 African languages. We focus on two questions: 1) How can pre-trained models be used for languages not included in the initial pretraining? and 2) How can the resulting translation models effectively transfer to new domains? To answer these questions, we create a novel African news corpus covering 16 languages, of which eight languages are not part of any existing evaluation dataset. We demonstrate that the most effective strategy for transferring both additional languages and additional domains is to leverage small quantities of high-quality translation data to fine-tune large pre-trained models.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Abstract: Recent advances in the pre-training for language models leverage large-scale datasets to create multilingual models. However, low-resource languages are mostly left out in these datasets. This is primarily because many widely spoken languages that are not well represented on the web and therefore excluded from the large-scale crawls for datasets. Furthermore, downstream users of [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_publishername":"","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"NAACL 2022","msr_number":"","msr_organization":"","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"","msr_doi":"","msr_arxiv_id":"","msr_s2_paper_id":"","msr_mag_id":"","msr_pubmed_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_original_fields_of_study":"","msr_release_tracker_id":"","msr_s2_match_type":"","msr_citation_count_updated":"","msr_published_date":"2022-5-3","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_s2_pdf_url":"","msr_year":0,"msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_match_confidence":0,"msr_microsoftintellectualproperty":true,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13545],"msr-publication-type":[193715],"msr-publisher":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[246808],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-848614","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-human-language-technologies","msr-locale-en_us","msr-field-of-study-natural-language-processing"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2022-5-3","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"NAACL 2022","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":1,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"file","viewUrl":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2022\/05\/a_few_thousand_translations_go.pdf","id":"848617","title":"a_few_thousand_translations_go","label_id":"243109","label":0}],"msr_related_uploader":"","msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[{"id":848617,"url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-content\/uploads\/2022\/05\/a_few_thousand_translations_go.pdf"}],"msr-author-ordering":[{"type":"text","value":"David Ifeoluwa Adelani","user_id":0,"rest_url":false},{"type":"text","value":"Jesujoba Oluwadara Alabi","user_id":0,"rest_url":false},{"type":"text","value":"Angela Fan","user_id":0,"rest_url":false},{"type":"text","value":"Julia Kreutzer","user_id":0,"rest_url":false},{"type":"text","value":"Xiaoyu Shen","user_id":0,"rest_url":false},{"type":"text","value":"Machel Reid","user_id":0,"rest_url":false},{"type":"text","value":"Dana Ruiter","user_id":0,"rest_url":false},{"type":"text","value":"Dietrich Klakow","user_id":0,"rest_url":false},{"type":"text","value":"Peter Nabende","user_id":0,"rest_url":false},{"type":"text","value":"Ernie Chang","user_id":0,"rest_url":false},{"type":"text","value":"Tajuddeen R. Gwadabe","user_id":0,"rest_url":false},{"type":"text","value":"Freshia Sackey","user_id":0,"rest_url":false},{"type":"text","value":"Bonaventure F. P. Dossou","user_id":0,"rest_url":false},{"type":"text","value":"Chris C. Emezue","user_id":0,"rest_url":false},{"type":"text","value":"Colin Leong","user_id":0,"rest_url":false},{"type":"text","value":"Michael Beukman","user_id":0,"rest_url":false},{"type":"text","value":"Shamsuddeen H. Muhammad","user_id":0,"rest_url":false},{"type":"text","value":"Guyo Dub Jarso","user_id":0,"rest_url":false},{"type":"text","value":"Oreen Yousuf","user_id":0,"rest_url":false},{"type":"text","value":"Andre Niyongabo Rubungo","user_id":0,"rest_url":false},{"type":"text","value":"Gilles Hacheme","user_id":0,"rest_url":false},{"type":"text","value":"Eric Peter Wairagala","user_id":0,"rest_url":false},{"type":"text","value":"Muhammad Umair Nasir","user_id":0,"rest_url":false},{"type":"text","value":"Benjamin A. Ajibade","user_id":0,"rest_url":false},{"type":"text","value":"Tunde O. Ajayi","user_id":0,"rest_url":false},{"type":"text","value":"Yvonne Wambui Gitau","user_id":0,"rest_url":false},{"type":"text","value":"Jade Abbott","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Maxamed Axmed","user_id":40888,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Maxamed Axmed"},{"type":"user_nicename","value":"Millicent Ochieng","user_id":40678,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Millicent Ochieng"},{"type":"text","value":"Anuoluwapo Aremu","user_id":0,"rest_url":false},{"type":"text","value":"Perez Ogayo","user_id":0,"rest_url":false},{"type":"text","value":"Jonathan Mukiibi","user_id":0,"rest_url":false},{"type":"text","value":"Fatoumata Ouoba Kabore","user_id":0,"rest_url":false},{"type":"text","value":"Godson Koffi Kalipe","user_id":0,"rest_url":false},{"type":"text","value":"Derguene Mbaye","user_id":0,"rest_url":false},{"type":"text","value":"Allahsera Auguste Tapo","user_id":0,"rest_url":false},{"type":"text","value":"Victoire M. Koagne","user_id":0,"rest_url":false},{"type":"text","value":"Edwin Munkoh-Buabeng","user_id":0,"rest_url":false},{"type":"text","value":"Valencia Wagner","user_id":0,"rest_url":false},{"type":"text","value":"Idris Abdulmumin","user_id":0,"rest_url":false},{"type":"text","value":"Ayodele Awokoya","user_id":0,"rest_url":false},{"type":"text","value":"Happy Buzaaba","user_id":0,"rest_url":false},{"type":"text","value":"Blessing Sibanda","user_id":0,"rest_url":false},{"type":"text","value":"Andiswa Bukula","user_id":0,"rest_url":false},{"type":"text","value":"Sam Manthalu","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[1021599],"msr_event":[],"msr_group":[643845],"msr_project":[1068003],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"article","related_content":{"projects":[{"ID":1068003,"post_title":"African Languages Research","post_name":"african-languages-research","post_type":"msr-project","post_date":"2024-08-14 06:39:23","post_modified":"2024-08-14 09:29:33","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/african-languages-research\/","post_excerpt":"Bridging technological and community divides by enriching Generative AI research with the rich diversity of African languages. Join us in shaping a more inclusive digital future Microsoft Research Africa is pioneering advancements in Generative AI through our African Languages Research initiative, focusing on advancing Large Language Models (LLMs) for African languages. Our goal is to enhance language accessibility and ensure AI tools can effectively engage with Africa's linguistic diversity, supporting both language preservation and the&hellip;","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/1068003"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/848614","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/848614\/revisions"}],"predecessor-version":[{"id":848623,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/848614\/revisions\/848623"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=848614"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=848614"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=848614"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=848614"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=848614"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=848614"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=848614"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=848614"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=848614"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=848614"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=848614"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=848614"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=848614"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}