The existing advanced super-resolution methods with deepening or widening network demand high computational resources and memory consumption. It is difficult to directly apply them in practice. Therefore, we propose a fast and lightweight two-channel end-to-end network with fewer parameters and low computational complexity in this paper. The shallow channel mainly restores the general outline of the image, while the deep channel mainly learns the high- frequency texture information. And the deep channel combines the dense block and residual connection. The dense block increases data flow of network, while the residual connection reduces the number of parameters and speeds up the convergence of network. Moreover, we propose an enhanced network using group convolution, which significantly reduces the parameters and computational complexity with slight performance loss. Our model has many fewer parameters and operations, and it is evaluated on different datasets, outperforming the current representative methods in accuracy and run time.