Many conventional demosaicking methods are based on hand-crafted filters. However, the filters yield false colors in salient regions like edges and textures. For acquisition of high quality images, we focus on neural networks. Neural networks lead to high accuracy in many fields. However, there are few methods in demosaicking field. For adaptation to demosaicking, we consider not only network's architecture but also the input. In this research, we utilize a Bayer image as input of our networks. However, different filter is needed in estimation at different color pixels, for example, missing red value at green pixel and that at blue pixel. Therefore, we prepare four networks with downsampling operators classified by color patterns in Bayer images. This downsampling operator not only identifies the color pattern but also reduces the calculation cost in each network due to reduction of the size of feature maps. Besides, preparation of multi-networks instead of a deep single-network is suitable for today's parallel computing. Moreover, we utilize not missing color images but chrominance images as output. Compared to results with missing color images as output, the results with chrominance images obtains higher accuracy. Experimental results show our CNN-based approach produces high quality restored images.