Most existing image denoising work assumes additive white Gaussian noise (AWGN) and removes the noise independent of the RGB channels. Therefore, the current approaches are not fully automatic and cannot effectively remove color noise produced by CCD digital camera. In this paper, we propose a framework for two tasks, automatically estimating and removing color noise from a single image using piecewise smooth image models. We estimate noise level function (NLF), a continuous function of noise level to image brightness, as the upper bound of the real noise level by fitting the lower envelope to the standard deviations of the per-segment image variance. In the denoising module, the chrominance of color noise is significantly removed by projecting pixel values to a line fit to the RGB space in each segment. Then, a Gaussian conditional random field (GCRF) is constructed to obtain the underlying clean image from the noisy input. Extensive experiments are conducted to test the proposed algorithms, which are proven to outperform the state-of-the-art denoising algorithms with promising and convincing results.