Color and depth information provided simultaneously in RGB-D images can be used to segment scenes into disjoint regions. In this paper, a graph-based segmentation method for RGB-D image is proposed, in which an adaptive data-driven combination of color- and normal-variation is presented to construct dissimilarity between two adjacent pixels and a novel region merging threshold exploiting normal information in adjacent regions is proposed to control the proceeding of the region merging. We evaluate our method on the NYUv2 depth database and compare it with several published RGB-D partition methods. The experimental results show that our method is comparable with the state-of-the-art methods and provides more details of structures in the scene.