Accurately tracking large tissue motion over a sequence of ultrasound images is critically important to several clinical applications including, but not limited to, elastography, flow imaging, and ultrasound-guided motion compensation. However, tracking in vivo large tissue deformation in 3D is a challenging problem and requires further developments. In this study, we explore a novel tracking strategy that combines Bayesian inference with local polynomial fitting. Since this strategy is incorporated into a region-growing block-matching motion tracking framework we call this strategy a Bayesian region-growing motion tracking with local polynomial fitting (BRGMTLPF) algorithm. More specifically, unlike a conventional block-matching algorithm, we use a maximum posterior probability density function to determine the “correct” three-dimensional displacement vector.The proposed BRGMT-LPF algorithm was evaluated using a tissue-mimicking phantom and ultrasound data acquired from a pathologically-confirmed human breast tumor. The in vivo ultrasound data was acquired using a 3D whole breast ultrasound scanner, while the tissue-mimicking phantom was acquired using an experimental CMUT ultrasound transducer. To demonstrate the effectiveness of combining Bayesian inference with local Polynomial fitting, the proposed method was compared to the original region-growing motion tracking algorithm (RGMT), region-growing with Bayesian interference only (BRGMT), and region-growing with local polynomial fitting (RGMT-LPF). Our preliminary data demonstrate that the proposed BRGMT-LPF algorithm can improve the accuracy of motion tracking.