Deep Metric Learning (DML) based on Convolutional Neural Networks (CNNs) is vulnerable to adversarial attacks. Adversarial training, where adversarial samples are generated at each iteration, is one of the prominent defense techniques for robust DML. However, adversarial training increases computational complexity and causes a trade-off between robustness and generalization. This study proposes a lightweight, robust DML framework that learns a non-linear projection to map the embeddings of a CNN into an adversarially robust space. The proposed method generates adversarial samples by attacking a pre-trained network once and employs them to learn a more robust mapping. Experiments on three well-known DML benchmark datasets show that the proposed lightweight approach can improve adversarial robustness while preserving natural performance.