Learning and Inferring a Semantic Space from User’s Relevance Feedback for Image Retrieval

As current methods for content-based retrieval are incapable of capturing the semantics of images, we experiment with using spectral methods to infer a semantic space from user’s relevance feedback, so the system will gradually improve its retrieval performance through accumulated user interactions. In addition to the long-term learning process, we also model the traditional approaches to query refinement using relevance feedback as a short-term learning process. Based on the model we derived a theoretical mistake upper bound which indicates that the maximum number of feedbacks is logarithmic with the total number of features and linear with the number of relevant features. The proposed short- and long-term learning frameworks have been integrated into an image retrieval system. Experimental results on a large collection of images have shown the effectiveness and robustness of our proposed algorithms.