Abstract. In this paper, we propose a novel variational energy formulation for automatic extraction of object of interest from images. Traditional variational energy formulation for image segmentation like that in  only incorporates local region potentials with a Gaussian distribution on each region. We argue that for segmentation of natural objects, Gaussian mixture model (GMM) needs to be adopted to capture the appearance variation of the objects. Moreover, we introduce a global image data likelihood potential to address the problem that each local region usually contains a portion of incorrectly partitioned pixels during the iterations. By combining it with local region potentials, we obtain more robust and accurate estimation of the foreground& background distributions. The minimization of the proposed local-global energy functional is achieved in two steps: the evolution of the foreground & background boundary curve by level set; and the robust estimation of the foreground& background model by fixed-point iteration called quasisemi-supervised EM, which is particularly suited for the learning problem where some unknown portion of the data are labeled incorrectly. Extensive experimental results including business card extraction, road sign extraction and general object-of-interest segmentation demonstrates the robustness, effectiveness, and efficiency of the proposed approach.