This paper proposes a novel unified framework for fault detection of the freight train images based on convolutional neural network (CNN) under complex environment. Firstly, the multi region proposal networks (MRPN) with a set of prior bounding boxes are introduced to achieve high quality fault proposal generation. And then, we apply a linear non-maximum suppression method to retain the most suitable anchor while removing redundant boxes. Finally, a powerful multi-level region-of-interest (ROI) pooling is proposed for proposal classification and accurate detection. The experimental results indicate that the proposed method can achieve high performance on four typical fault benchmarks, substantially outperforming the state-of-the-art methods.