This paper presents a simple yet effective method to improve the visual quality of Generative Adversarial Network (GAN) generated images. In typical GAN architectures, the discriminator block is designed mainly to capture the class-specific content from images without explicitly imposing constraints on the visual quality of the generated images. A key insight from the image quality assessment literature is that natural scenes possess a very unique local structural and (hence) statistical signature, and that distortions affect this signature. We translate this insight into a constraint on the loss function of the discriminator in the GAN architecture with the goal of improving the visual quality of the generated images. Specifically, this constraint is based on the Multi-scale Structural Similarity (MS-SSIM) index to guarantee local structural and statistical integrity. We train GANs (Boundary Equilibrium GANs, to be precise,) using the proposed approach on popular face and car image databases and demonstrate the improvement relative to standard training approaches both visually and quantitatively.