High quality algorithms are demanded to reconstruct distorted color images in a variety of applications. For example, distortions can result during transmission over lossy channels in image coding or in multi-view imaging scenarios. In general, not all color channels are equally affected and the losses distribute differently in-between channels. However, state-of-the-art methods process color channels independently and do not take the cross color information into account. Thus, a novel and powerful reconstruction algorithm is formulated in this contribution that exploits color as well as spatial information. Therefore, an initial model is estimated for the distorted area using a reference channel. Then, its quality is estimated and a spatial weighting model is set-up. Afterwards, the initial inter channel prediction is refined by generating a sparse model that takes the spatial correlations into account, as well. Consequently, the proposed method achieves an outstanding quality compared to state-of-the-art methods.