Fetal well-being during labor is currently assessed by medical professionals through visual interpretation of the cardiotocogram (CTG), a simultaneous recording of Fetal Heart Rate (FHR) and Uterine Contractions (UC). This method is disputed due to high inter- and intra-observer variability and a resulting increase in the number of unnecessary interventions. A method for computerized interpretation of the CTG, based on Contrastive Predictive Coding (CPC) is presented here. We hypothesize that the CPC framework, when trained on healthy fetuses only, can predict the FHR response of healthy fetuses to UC, but will provide significant prediction error in case of fetuses with compromised condition. To that end, we have extended the original CPC model by making stochastic, recurrent, and conditioned (upon Uterine Contractions) predictions. We, moreover, introduce a new training objective that was found more suitable for the task of anomaly detection. Based on the detection of out-of-distribution behaviour and deviations from subject-specific behaviour, the proposed model is capable of achieving promising results for identification of suspicious and anomalous FHR events in the CTG, with an average correlation coefficient of 0.80±0.13 with respect to expert annotations.