The 3D extension of High Efficiency Video Coding (3D-HEVC) achieves excellent performance for 3D video coding while possessing significant computational complexity. To accelerate the time-consuming coding process of the depth map, a fast algorithm via XGBoost is proposed in this paper. Specifically, a total of 14 specialized XGBoost models are used for different block sizes and viewpoint types to achieve early coding unit partition determination (ECP) and early prediction unit mode selection (EPM) to avoid executing the exhaustive traversal coding process. To promote the prediction accuracy of XGBoost models, multi-domain correlations, including spatiotemporal, inter-view, and inter-component correlations are utilized and plenty of features are selected for model training. Evaluated on HTM-16.0 under random access configuration, the proposed ECP strategy can obtain 51.2% total encoding time saving with a 0.18% BDBR increase and the ECP+EPM can overall achieve 60.8% total encoding time saving with a 0.59% BDBR increase. The source code of our method is available at https://github.com/Joeyrr/ECP_EPM.git.