Although promising results have been achieved in the areas of object detection and classification, few works have provided an end-to-end solution to the perception problems in the autonomous driving field. In this paper, we make two contributions. Firstly, we fully enhanced our previously released TT100K benchmark and provide 16,817 elaborately labeled Tencent Street View panoramas. This newly created benchmark, we call it Tencent Autonomous Driving 16K (TAD16K), not only contains previously labeled traffic-signs (221 types), but also creates annotations for three new objects, which are traffic lights (6 types), vehicles and pedestrians. Secondly, we provide the evaluation results of two state-of-the-art object detection algorithms (SSD and DetectNet) on our benchmark, which can be used as the baseline for future comparison purpose. Finally, we also demonstrate that the network trained on our benchmark can be directly deployed for practical application. The TAD16K, relevant additions and the source codes are publicly available.(1.TAD16K: http://autopilot.qq.com/ICIP2017/. 2.Source code: https://github.com/lymhust/TAD16K_source.)