Group sparsity or nonlocal image representation has shown great potential in image denoising. However, most existing methods only consider the nonlocal self-similarity (NSS) prior of noisy input image, that is, the similar patches collected only from degraded input, which makes the quality of image denoising largely depend on the input itself. In this paper we propose a new prior model for image denoising, called group sparsity residual constraint (GSRC). Different from the most existing NSS prior-based denoising methods, two kinds of NSS prior (i.e., NSS priors of noisy input image and pre-filtered image) are simultaneously used for image denoising. In particular, to boost the performance of group sparse-based image denoising, the group sparsity residual is proposed, and thus the problem of image denoising is transformed into one that reduces the group sparsity residual. To reduce the residual, we first obtain a good estimation of the group sparse coefficients of the original image by pre-filtering and then the group sparse coefficients of noisy input image are used to approximate the estimation. To improve the accuracy of the nonlocal similar patches selection, an adaptive patch search scheme is proposed. Moreover, to fuse these two NSS priors better, an effective iterative shrinkage algorithm is developed to solve the proposed GSRC model. Experimental results have demonstrated that the proposed GSRC modeling outperforms many state-of-the-art denoising methods in terms of the objective and the perceptual qualities.