MindFinder: Finding Images by Sketching

Established: August 12, 2010

Sketch-based image search is a well-known and difficult problem, in which little progress has been made in the past decade in developing a large-scale and practical sketch-based search engine. We have revisited this problem and developed a scalable solution to sketch-based image search. The MindFinder system has been built by indexing more than two million web images to enable efficient sketch-based image retrieval, and many creative applications can be expected to advance the state of the art.

Project Descriptions

Sketch-based image search is a well-known problem difficult to solve. Little progress has been made in the past decade in developing a large-scale, practical, sketch-based search engine. We have revisited this problem and developed a scalable solution. Based on this solution, we have built a system called MindFinder that indexes more than 2 million web images, enabling efficient, sketch-based image retrieval. MindFinder enables users to sketch major curves of a target image as a query. Based on these sketches, relevant images can be returned in real time. The system also supports the addition of semantic tags and colors to enhance the relevance of search results.

Videos 

Softwares

Sketch Match: On 2011-08-17, the app, Sketch Match, was released to WP7 marketplace! It is powered by the MindFinder technology (sketch-based image search), with a well-refined sketch matching game.

 

Sketch Match is powered by MindFinder research work. The app supports users to search for similar images by sketching. It also provides a sketch matching game for entertainment and learning.

Use your fingers to sketch what you’re imagining and find similar images:

screenshot7.jpg screenshot8.jpg

In the matching game, sketch the images as closely as possible to score:

screenshot2.jpg screenshot3.jpg

Find your sketching rank in the global leaderboard:

screenshot9.jpg

Publications

  •  Xinghai Sun, Changhu Wang, Avneesh Sud, Chao Xu, Lei Zhang. MagicBrush: Image Search by Color Sketch (Demo). Accepted by ACM Multimedia 2013.
  • Xinghai Sun, Changhu Wang, Chao Xu, Lei Zhang. Indexing Billions of Images for Sketch-based Retrieval. Accepted by ACM Multimedia 2013.
  • Jiangping Wang, Changhu Wang, Thomas Huang. Efficient Image Contour Detection using Edge Prior”. ICME 2013.
  • Zhenbang Sun, Changhu Wang, Liqing Zhang, Lei Zhang, Sketch2Tag: Automatic Hand-Drawn Sketch Recognition (Demo), ACM Multimedia 2012.
  • Zhenbang Sun, Changhu Wang, Liqing Zhang, Lei Zhang, Query-Adaptive Shape Topic Mining for Hand-Drawn Sketch Recognition, in ACM Multimedia 2012.
  • Zhenbang Sun, Changhu Wang, Liqing Zhang, Lei Zhang, Free Hand-Drawn Sketch Segmentation, in ECCV 2012.
  • Xianming Liu, Changhu Wang, Hongxun Yao, Lei Zhang, The Scale of Edges, in CVPR 2012.
  • Changhu Wang, Jun Zhang, Bruce Yang, Lei Zhang, Sketch2Cartoon: Composing Cartoon Images by Sketching, in ACM Multimedia 2011.
  • Wei Zheng, Changhu Wang, Xilin Chen, Shape-based Web Image Clustering for Unsupervised Object Detection, in ICME 2011.
  • Changhu Wang, Yang Cao, Lei Zhang, MindFinder: A Sketch-based Image Search Engine based on Edgel Index (demo), in CVPR 2011.
  • Yang Cao, Changhu Wang, Liqing Zhang, and Lei Zhang, Edgel Inverted Index for Large-Scale Sketch-based Image Search, in CVPR 2011.
  • Yang Cao, Hai Wang, Changhu Wang, Zhiwei Li, Liqing Zhang, and Lei Zhang, MindFinder: Interactive Sketch-based Image Search on Millions of Images, in ACM Multimedia 2010 International Conference, 2010. (Best demo award)
  • Changhu Wang, Zhiwei Li, and Lei Zhang, MindFinder: Image Search by Interactive Sketching and Tagging, in WWW’10: 19th International World Wide Web Conference, 2010.

People

Publications