Camera pose estimation plays a crucial role in stitching overlapped images captured by a camera to achieve a broad view of interest. In this paper, we propose a robust camera pose estimation approach to stitching images of a large 3D surface with known geometry. In particular, given a collection of images, we first construct a relative pose matrix estimation of all image pairs from the collection, where each entry of the matrix is calculated by solving a perspective-n-point(PnP) problem over the corresponding pair of images. To continue, we jointly estimate all camera poses by solving an optimization problem that exploits the underlying rank-2 relative pose matrices and the joints parsity of camera pose errors. Finally, images are projected onto the 3D surface of interest based on estimated camera poses for the subsequent stitching process. Numerical experiments demonstrate that our proposed method outperforms existing methods in terms of both camera pose estimation and image stitching quality