Multimedia Search and Mining

Established: November 18, 2013

Multimedia Search and Mining (MSM) group focuses on a wide variety of multimedia-related research and projects, e.g., understanding, analysis, search, data mining, and applications. We are working on research problems in image understanding, video analytics, large scale visual (image and video) indexing and search, 3D reconstruction, and so on.

People

Publications

Videos

Projects

Deep Neural Networks

Established: September 1, 2015

We study how to morph a well-trained neural network to a new one, and how to design advanced deep neural networks.

Video Analytics

Established: March 16, 2016

Video has become ubiquitous on the Internet, broadcasting channels, as well as that captured by personal devices. This has encouraged the development of advanced techniques to analyze the semantic video content for a wide variety of applications, such as video…

Image chat

Established: February 22, 2016

Image is becoming a popular media for user communications on social networks. Then, it comes to be a natural requirement to enable chatbot to chat on images besides textual inputs. Based on MS XiaoIce(微软小冰), we explore the direction of image…

Photo Story

Established: January 25, 2016

The capability of managing personal photos is becoming crucial. In this work, we have attempted to solve the following pain points for mobile users: 1) intelligent photo tagging, best photo selection, event segmentation and album naming, 2) speech recognition and…

Image/Video Understanding and Analysis

Established: February 1, 2014

We target at the core problems in image/video understanding and analysis, such as image recognition, image segmentation, image captioning, image parsing, object detection, and video segmentation.

Food Recognition

Established: January 25, 2016

  We study the problem of food image recognition via deep learning techniques. Our goal is to develop a robust service to recognize thousands of popular Asia and Western food. Several prototypes have been developed to support diverse applications. The…

Video and Language

Established: January 14, 2016

Automatically describing video content with natural language is a fundamental challenge of computer vision. Recurrent Neural Networks (RNNs), which models sequence dynamics, has attracted increasing attention on visual interpretation. In this project, we present a novel unified framework, named Long…

MindFinder: Finding Images by Sketching

Established: August 12, 2009

Sketch-based image search is a well-known and difficult problem, in which little progress has been made in the past decade in developing a large-scale and practical sketch-based search engine. We have revisited this problem and…

Mobile Video Search

Established: February 17, 2014

Mobile video is quickly becoming a mass consumer phenomenon. More and more people are using their smartphones to search and browse video contents while on the move. This project is to develop an innovative instant mobile video search system through…

Picto: A large scale visual indexing and recognition system

Established: September 1, 2009

Object image recognition is a challenge but important problem. Towards addressing this problem, we initialed the Picto project. Our research in this project covers three fundamental aspects of this problem: low-level image features, middle level image representations, and indexing and…