Prior to joining Microsoft, he also conducted research and taught at Massachusetts Institute of Technology, ATR Interpreting Telecommunications Research Lab. (Kyoto, Japan), and HKUST. He has been granted over 70 US or international patents in acoustics/audio, speech/language technology, large-scale natural language and enterprise/internet data analysis, and in machine learning with recent focus on deep learning. He received numerous awards/honors bestowed by IEEE, International Speech Communication Association, Acoustical Society of America, Asia-Pacific Signal & Information Processing Association, Microsoft, and other organizations.
His current (and past) research activities include deep learning and machine intelligence applied to big text data and to speech, image and multimodal processing, computational neuroscience and information representation, deep/recurrent/dynamic neural networks, automatic speech and speaker recognition, spoken language identification and understanding, speech-to-speech translation, machine translation, language modeling, information retrieval and data mining, web search, neural information processing, dynamic systems, machine learning and optimization, parallel and distributed computing, probabilistic graphical models, audio and acoustic signal processing, image analysis and recognition, compressive sensing, statistical signal processing, digital communication, human speech production and perception, acoustic phonetics, auditory speech processing, auditory physiology and modeling, noise robust speech processing, speech synthesis and enhancement, multimedia signal processing, and multimodal human-computer interactions.
In the general areas of audio/speech/language technology and science, machine learning, signal/information processing, and other areas of computer science, he has published over 300 refereed papers in leading journals and conferences, and authored or co-authored 5 books including the latest books on Deep Learning: Methods and Applications and on Automatic Speech Recognition: A Deep-Learning Approach (Springer). He is a Fellow of the Acoustical Society of America, a Fellow of the IEEE, and a Fellow of the International Speech Communication Association. He served on the Board of Governors of the IEEE Signal Processing Society (2008-2010), and as Editor-in-Chief for the IEEE Signal Processing Magazine (2009-2011), which earned the highest impact factor in 2010 and 2011 among all IEEE publications and for which he received the 2012 IEEE SPS Meritorious Service Award. Most recently, he served as General Chair of the IEEE ICASSP-2013, and as Editor-in-Chief for the IEEE Transactions on Audio, Speech and Language Processing (2012-2014). His technical work since 2009 (when he initiated deep learning research and technology development at Microsoft with Geoff Hinton) and the leadership in industry-scale deep learning with colleagues have created high impact in speech recognition and other areas of information processing. The work by him and the team he manages has been in use in major Microsoft speech and text/data-related products, and is recognized by IEEE SPS Best Paper Award, IEEE Outstanding Engineer Award, APSIPA Industrial Distinguished Leader Award, Microsoft Goldstar and Technology Transfer Awards.
His recent research interests and activities have been focused on deep learning and machine intelligence applied to large-scale text analysis and to speech/language/image multimodal processing, advancing his earlier work on speech analysis/recognition using deep neural networks and deep generative models.