Microsoft Research Blog

The Microsoft Research blog provides in-depth views and perspectives from our researchers, scientists and engineers, plus information about noteworthy events and conferences, scholarships, and fellowships designed for academic and scientific communities.

Microsoft researchers tie for best image captioning technology

June 11, 2015 | By Microsoft blog editor

Posted by Allison Linn

Researchers representing Microsoft and Google will present their latest advances Friday in automated image captioning, a hot field that could have broad implications for artificial intelligence.

The researchers will be speaking at a workshop that is part of CVPR, an annual conference on the most cutting-edge advances in computer vision research. The workshop is highlighting the winners of several image-related challenges.

The two companies’ research groups tied for first place in the recent MS COCO Image Captioning Challenge 2015. There were 15 submissions from top universities and industrial research labs vying to automatically create the most informative and interesting captions.

The winners were decided based on two main metrics: The share of captions that were equal to or better than a caption written by a person, and the share of captions that would pass a Turing test.

The Turing test, named after a paper published by Alan Turing in 1950, is a test of whether a human would believe something generated by a computer was actually written by a human.

The Microsoft team outperformed competitors on the Turing test element, while the Google team won for the share of captions that were as good, or better, than what people could produce.

The field of automated image captioning has exploded since researchers hit upon the idea of using neural networks, which are computing elements that are modeled loosely after the human brain, to connect vision to language.

Many researchers see image captioning as the basis for more sophisticated artificial intelligence systems that can see, hear, speak and even understand.


Allison Linn is a senior writer at Microsoft Research. Follow Allison on Twitter.