To investigate the temporal dynamics and spatial representations in speech perception and processing, electroencephalograph (EEG) signals were recorded when subjects listening Chinese words and pseudo words. According to the source reconstruction, the early neural activations were observed in wider areas from the Spt (an area in the Sylvian fissure at the parietotemporal boundary) to the premotor cortex and posterior frontal gyrus during 35ms to 80ms, while the areas from posterior middle temporal gyrus to anterior middle temporal gyrus were activated during a time period from 110ms to 180ms. These activation patterns are consistent with the dual-stream model, where the former one corresponds to the dorsal stream and the latter one belongs to the ventral stream. These two streams interacted after 180ms and showed recurrent activation after 300ms. This phenomenon was not observed for the pseudo words. Our findings demonstrate a quite early stage of speech cognitive process and present the dynamics of interaction between the two streams during perceiving words. The results are essential for the construction of connectivity model in further studies.