Cross-modal retrieval is a hot topic in the fields of machine learning and media retrieval, making it possible to relate different types of media, such as image, text, and audio. A powerful method for the cross-modal retrieval, discrete cross-modal hashing (DCH), has recently been proposed. The DCH can encode different types of media feature vectors to binary codes. When stored in a database, the binary code makes searches efficient because the Hamming distance between the corresponding sections of two binary codes can be computed via a specialized CPU operation. Moreover, it has recently been shown that when optimizing hash functions for supervised discrete hashing (SDH), Hadamard matrices can be used, and this technique is named “HC-SDH”. In this study, we apply the HC-SDH to cross-modal hashing. Experimental results demonstrate that the proposed cross-modal hashing can achieve the same performance as the conventional DCH with 1/24 of the training time.