Speech discourse comprehension is crucial for developing intelligent speech processing technologies. The present research aims to establish a multi-layered annotation scheme for Chinese discourse that contains inter-related information of phonetics, phonology, syntax, semantics and pragmatics. This research provides a theoretical foundation and analytical support for discourse comprehension by examining and modelling the relationships between prosody and morphology-syntax, as well as semantics and other structures during speech interactions. This research recognizes that prosodic research on different discourse levels is of a great significance due to the unique characteristics of Chinese discourse. Specifically, we propose a hierarchical representation structure and correspondingly an annotation convention for Chinese discourse, namely HiSAC (Hierarchical Scheme and Annotation Convention for Chinese Discourse). Based on the scheme,an annotated speech corpus is constructed. Finally, three case studies are presentedto demonstrate the usefulness of the proposed scheme and corpus in interfacing the prosody and speech discourse,with respect to the relationships between prosodic features and dependency structure, information structure, rhetorical structure respectively.