Due to the high computation complexity and intra-class variance in the area of image pattern recognition, feature extraction for image pattern recognition has been the focus of interest for quite some time. In this paper, a novel feature extraction framework is presented, which first constructs an over-complete feature combination set, and then selects effective combinations by using feature selection algorithm. Experimental results show that this structure can do pattern recognition on images more efficiently in both accuracy and speed.