Soft-biometric privacy enhancing techniques (SB-PETs) transform facial images to preserve identity while preventing the automatic extraction of soft-biometrics by confusing machines through noise injections or attribute obfuscation. However, existing SB-PETs often sacrifice image quality for privacy enhancement, limiting practical usage, especially in applications that allow for human inspection. To address these issues, we introduce a novel SB-PET that (i) generates photo-realistic images with obscured gender information, which makes attribute extraction challenging for machine-learning models, but also human observers, and (ii) preserves identity to a significant extent. The proposed approach, abbreviated PriDSS, operates in the latent space of the StyleGANv2 model and aims to (i) preserve the appearance of facial parts from the input image carrying identity information, and (ii) incorporate global context from images of the opposite gender, thus, obscuring the original gender information. PriDSS shows promising results when compared to state-of-the-art techniques from the literature, and leads to competitive gender-privacy and face-verification performance, while ensuring superior photo-realism.