Microsoft AI for Accessibility Sign Language Recognition & Translation Workshop

Abstract: This introduction will briefly cover all elements of Deaf culture, language, accessibility, and cultural competence.  With so many products and technology developed out there by non-Deaf people, the Deaf community is struggling with a “disability designation” when they face barriers that could have been avoided if the developers took Deaf people in consideration for universal design to keep our world accessible to all people and empowering to Deaf people themselves.

Abstract: Language is a technology of the human body. Languages emerge and evolve to most efficiently communicate what their users need, can use and will use. Originating in deaf communities around the world, sign languages exploit the capacities of the visual-manual modality and the grammatical use of body-space configurations for communicative function. Users of sign language, however, are a diverse group of individuals, especially with respect to hearing status and linguistic competence and experience. This diversity impacts how sign languages are used in the practice of everyday life, which one needs to account for in designing sign language tools. This talk covers the key elements of sign language structure that are relevant for designing language technologies involving sign language and their users.  

Abstract: The field of natural language processing (NLP) is now delivering powerful technologies: computers can translate from one language to another, answer questions, and hold conversations.  In this relatively non-technical talk, I will trace the evolution of one aspect of NLP programs: how we put English (or other-language) words into computers.  While this talk won’t teach you everything about NLP, it will illuminate some of its toughest challenges and most exciting recent advances. 

Abstract: This talk will present recent advances in the field of sign language recognition observed from an interdisciplinary view point at the intersection of speech recognition and computer vision. We will show several examples, analyze available data sets and trending methods. We will also understand what challenges remain to be tackled.

Abstract: There is great diversity in the levels of English reading skill and in the language preferences among members of the U.S. Deaf Community, and many individuals prefer to receive information in the form of American Sign Language (ASL). Therefore, providing ASL on websites can make information and services more accessible. Unfortunately, video recordings of human signers are difficult to update when information changes, and there is no way to support just-in-time generation of website content from a user request. Software is needed that can automatically synthesize understandable animations of a virtual human performing ASL, based on an easy-to-update script as input. The challenge is for this software to select the details of such animations so that they are linguistically accurate, understandable, and acceptable to users.  This talk will provide an overview of Huenerfauth’s research in using machine-learning techniques to model human movements. His methodology includes: video and motion-capture data collection from signers to collect a corpus of ASL, linguistic annotation of this corpus, statistical modeling techniques, animation synthesis, and experimental evaluation studies with native ASL signers. In this way, his laboratory has found models that underlie the accurate and natural movements of virtual human characters performing ASL. In recent work, his laboratory has created models to predict essential speed and timing parameters for such animations. 

Panelists: Michael Anthony, Lance Forshay, Leah Katz-Hernandez, Christian Vogler