Current state-of-the-art music boundary detection methods use local features for boundary detection, but such an approach fails to explicitly incorporate the statistical properties of the detected segments. This paper presents a music boundary detection method that simultaneously considers a fitness measure based on the boundary posterior probability, the likelihood of the segmentation duration sequence, and the acoustic consistency within a segment. Evaluation shows that our method improves segmentation F0.58-measure by about 10 points compared to DNN with peak-picking, a popular scheme used in the state-of-the-art music boundary detectors.