Due to the rich details of residential areas and the characteristics of remote sensing image sharpness vulnerable to haze, it will not only consume a lot of labor costs but also be very difficult to produce a large-scale dataset with strong labels. Therefore, the limited-sample dataset has become a hotspot in recent years. To address this issue, we proposed a semantic segmentation method for residential areas by phase learning. The main task of the first stage is to generate a joint saliency map by reducing the interference of haze noise through the feature comparison similarity sorting algorithm and combine them to generate initial pixel-level pseudo labels for the next stage of training. In the second stage, we proposed to construct a group feature interactive perception module to achieve image group semantic co-segmentation. Comprehensive evaluations with 2 datasets and the comparison with 7 methods validate the superiority of the proposed model.