Following the significance of soft biometrics to facilitate seamless recognition or retrieval, the need for multi-modality annotated datasets is increasing - to evaluate any standalone soft biometrics system. Although, large-size datasets like PETA were annotated to evaluate soft biometrics systems, however, they were mainly annotated for global soft biometrics such as gender and age and for clothing modality. By looking at the usefulness of multiple modalities of the human body during recognition or retrieval, we designed, developed and annotated a new dataset called Annotated Pedestrians for the individuals. The images in the dataset were explicitly recorded for the individuals at four different distances from the camera and they incorporate annotations for four different modalities of the human body i.e., i) global soft biometrics, ii) extended facial region, iii) body including limbs, and iv) clothing with attachments. The annotation process was expert opinion and qualitative annotation types were used. There were a total of three global soft biometrics annotated and for remaining three modalities, categorical annotations for 46 soft biometrics were performed. In terms of comparative annotations, there were a total of 26 soft biometrics annotated for the same three modalities. To the best of our knowledge, Annotated Pedestrians is a unique dataset designed by incorporating the impact of distance during recognition or retrieval, where markers were placed on the surface at 4, 6, 8, and 10 $m$ distances from the camera, and approximately 300 frames were recorded for 50 distinct individuals in a 20 $m$ long corridor. Moreover, the usefulness of the dataset is annotation using four different modalities of the human body, and a total of 75 soft biometrics annotated using a qualitative approach - making Annotated Pedestrians a highly-diverse dataset to evaluate any soft biometrics system for recognition during short-term tracking and feature-based retrieval from the database.