At present, the performance of the end-to-end stereo matching networks based on CNN greatly exceed the traditional stereo matching networks, but the accuracy in those ill-posed regions like foreground areas is still not optimistic. In this paper, we propose a novel design to improve the prediction performance of disparity in foreground. First, a multi-scale pyramid aggregation module with hourglass-like structure is designed to effectively utilize the aggregation information of different scales. In addition, we propose MPANet, a novel end-to-end stereo matching network, which significantly alleviates the disparity mismatch problem in foreground areas. Experimental results on Scene Flow and KITTI datasets also demonstrate that our network has competitive performance among existing first-class methods.