Video summarization considers the problem of selecting a concise set of frames or shots to preserve the most essential contents of the original video. Most of the current approaches apply Recurrent Neural Network (RNN) to learn the interdependencies among the video frames without considering the distinct information of particular frames. Other methods leverage the attention mechanism to explore the characteristics of some certain frames, while ignoring the systematic knowledge across the video sequence. In this paper, a novel video summarization methodology, Wide and Deep Summarization Network (WD-SN), is proposed based on attention mechanism and Independently Recurrent Neural Network(IndRNN). It captures both the wide independent characteristics and the deep interdependencies of a video sequence. Experiments on SumMe and TVSum datasets demonstrate the effectiveness of our framework.