Generating Descriptions of Visible Objects

In this talk, I will be discussing the connection between vision and language, particularly focusing on how to generate human-like reference to visible objects. Some of the findings in this talk have been used to automatically generate descriptions of images (EACL 2012), order descriptive modifiers before a noun (ACL 11), and approximate human preferences for describing color and size (ENLG 2011, NAACL 2013). Evaluating automatically generated text offers interesting challenges, particularly when aiming to capture speaker variation, and I will spend some time discussing how best to measure the naturalness of generated descriptions against a corpus of human-produced descriptions.

Speaker Details

Margaret Mitchell is a postdoctoral researcher at the Johns Hopkins University HLT Center of Excellence, working on sentiment analysis and named entity recognition using graphical models. She completed her Ph.D. in 2013 from the University of Aberdeen, researching the generation of reference to visible objects with Kees van Deemter and Ehud Reiter. She has also worked on assistive language generation technology and automatically diagnosing neurological disorders with Brian Roark and Richard Sproat at Oregon Health and Science University. She received a master’s degree in computational linguistics from the University of Washington and an undergraduate degree in Linguistics from Reed College.

Relevant papers:
Mitchell, M. and van Deemter, K. and Reiter, E. (2013). Generating Expressions that Refer to Visible Objects. Proceedings of NAACL 2013.
Mitchell, M., Dodge, J., Goyal, A., Yamaguchi, K., Stratos, K., Han, X., Mensch, A., Berg, A., and Berg, T. L., Daume III, H. (2012). Midge: Generating Image Descriptions From Computer Vision Detections. Proceedings of EACL 2012.
Mitchell, M., van Deemter, K., and Reiter, E. (2011). Two Approaches for Generating Size Modifiers. Proceedings of ENLG 2011.
Mitchell, M., Dunlop, A., and Roark, B. (2011). Semi-Supervised Modeling for Prenominal Modifier Ordering. Proceedings of ACL 2011.

Date:
Speakers:
Margaret Mitchell
Affiliation:
Johns Hopkins University

    • Portrait of Jeff Running

      Jeff Running

    • Portrait of Margaret Mitchell

      Margaret Mitchell

      Researcher