About

Partner Research Manager in Deep Learning Technology Center (DLTC) at Microsoft Research, Redmond. I work on deep learning for text and image processing (see our JICAI2016 Tutorial or MS internal site) and lead the development of AI systems for dialogue, machine reading comprehension (MRC), question answering (QA), and enterprise applications.

We are hiring Researchers with strengths in ML and NLP, and Software Engineers with rich product experience.

DSSM: We have developed a series of deep semantic similarity models (DSSM, also a.k.a. Sent2Vec), which have been used for many text and image processing tasks, including web search [Huang et al. 2013, Shen et al. 2014], recommendation [Gao et al. 2014a], machine translation [Gao et al. 2014b], and QA [Yih et al. 2015].

MRC: We released a new MRC dataset, called MS MARCO; and have developed a series of reasoning networks for MRC, aka ReasoNet and ReasoNet with shared memory.

Dialogue: We have developed neural network models for social bots trained on Twitter data [project site] and task-completion bots [Lipton et al. 2016; Bhuwan et al. 2016] trained via reinforcement learning using a user simulator.

From 2006 to 2014, I was Principal Researcher at Natural Language Processing Group at Microsoft Research, Redmond. I worked on Web search, query understanding and reformulation, ads prediction, and statistical machine translation.

From 2005 to 2006, I was a research lead in Natural Interactive Services Division at Microsoft. I worked on Project X, an effort of developing natural user interface for Windows.

From 1999 to 2005, I was Research Lead in Natural Language Computing Group at Microsoft Research Asia. I, together with my colleagues, developed the first Chinese speech recognition system released with Microsoft Office, the Chinese/Japanese Input Method Editors (IME) which were the leading products in the market, and the natural language platform for Windows Vista.

Currently, I live with my family in Woodinville, WA.

Projects

MS-Celeb-1M: Challenge of Recognizing One Million Celebrities in the Real World

Established: June 29, 2016

MSR Image Recognition Challenge (IRC) @ACM Multimedia 2016 Latest Updates: Participants information disclosed in "Team Information" section below 6/21/2016: Evaluation Result Announced in "Evaluation Result " section below. 6/17/2016: Evaluation finished. 14 teams finished the grand challenge! 6/13/2016: Evaluation started. 6/13/2016: Dry…

From Captions to Visual Concepts and Back

Established: April 9, 2015

We introduce a novel approach for automatically generating image descriptions. Visual detectors, language models, and deep multimodal similarity models are learned directly from a dataset of image captions. Our system is state-of-the-art on the official Microsoft COCO benchmark, producing a…

Data-Driven Conversation

Established: June 1, 2014

This project aims to enable people to converse with their devices. We are trying to teach devices to engage with humans using human language in ways that appear seamless and natural to humans. Our research focuses on statistical methods by…

MSR SPLAT

Established: April 4, 2012

Statistical Parsing and Linguistic Analysis Toolkit is a linguistic analysis toolkit. Its main goal is to allow easy access to the linguistic analysis tools produced by the Natural Language Processing group at Microsoft Research. The tools include both traditional linguistic…

Microsoft Research ESL Assistant

Established: May 9, 2008

The Microsoft Research ESL Assistant is a web service that provides correction suggestions for typical ESL (English as a Second Language) errors. Such errors include, for example, the choice of determiners (the/a) and the choice…

Publications

2017

2016

2015

From Captions to Visual Concepts and Back
Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh Srivastava, Li Deng, Piotr Dollar, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John Platt, Larry Zitnick, Geoffrey Zweig, in The proceedings of CVPR, IEEE – Institute of Electrical and Electronics Engineers, June 1, 2015, View abstract, Download PDF

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Projects

Link description

Deep Learning for Text Processing

Date

August 4, 2014

Speakers

Li Deng, Eric Xing, Xiaodong He, Jianfeng Gao, Christopher Manning, Paul Smolensky, and Jeff A Bilmes

Affiliation

MSR, Carnegie Mellon University, Microsoft Research, Redmond, MSR Redmond, Stanford, Johns Hopkins University, University of Washington

Link description

UW/MS symposium

Date

June 6, 2008

Speakers

Danyel Fisher, Douglas Downey, Chris Quirk, Scott Drellishak, Kelly O'Hara, Emily M. Bender, Sumit Basu, Matthew Hurst, Arnd Christian König, Michael Gamon, Chris Brockett, Dmitriy Belenko, Bill Dolan, Jianfeng Gao, and Lucy Vanderwende

Downloads

Scalable Language-Model-Building Tool

October 2010

This scalable language-model tool is used to build language models from large amounts of data. It supports modified absolute discounting and Kneser-Ney smoothing. The tool has been used successfully to build a seven-gram language model on 40 billion words within eight hours.

Size: 11 MB

    Click the icon to access this download

  • Website

Bayesian Estimators for Unsupervised HMM Part-of-Speech Tagger

August 2009

    Click the icon to access this download

  • Website

S-MSRSeg

February 2008

    Click the icon to access this download

  • Website

NLP Data Sets for Comparative Study of Parameter-Estimation Methods

June 2007

    Click the icon to access this download

  • Website