Classification of plants based on a multi-organ approach is very challenging. Although additional data provides more information that might help to disambiguate between species, the variability in shape and appearance in plant organs also raises the degree of complexity of the problem. Existing approaches focus mainly on generic features for species classification, disregarding the features representing the organs. In fact, plants are complex entities sustained by a number of organ systems. In our approach, we exploit the PlantClef2015 benchmark, and introduce a hybrid generic-organ convolutional neural network (HGO-CNN), which takes into account both organ and generic information, combining them using a new feature fusion scheme for species classification. We show that our proposed method outperforms the state-of-the-art results.