This paper shows the benefits of using Complex-Valued Neural Network (CVNN) on classification tasks for non-circular complex-valued datasets. Motivated by radar and especially Synthetic Aperture Radar (SAR) applications, we propose a statistical analysis of fully connected feed-forward neural networks performance in the cases where real and imaginary parts of the data are correlated through the non-circular property. In this context, comparisons between CVNNs and their real-valued equivalent models are conducted, showing that CVNNs provide better performance for multiple types of non-circularity. Notably, CVNNs statistically perform less overfitting, higher accuracy and provide shorter confidence intervals than its equivalent Real-Valued Neural Network (RVNN).