Recently, sparse representation based classification has been widely used in pattern recognition. Most of existing methods exploit the recovered representation coefficients to reconstruct the inputs, and the classwise reconstruction errors are used to identify the class of the sample based on the subspace assumption. Different from the reconstruction pipeline, an assignment framework is built on the representation coefficients in this paper. More specifically, we treat the representation coefficients as soft assignments of the class labels, and the distribution of the assignments reveals the class of the sample. Under this framework, we can easily generalize it to multi-sample and/or multi-feature scenarios, where multiple assignment instances can be directly fused to stabilize the distribution estimation. As such, the estimated distribution pattern can be used as a new discriminative feature for classification. Experiments on the classification of hyperspectral image demonstrate that the generalized assignment framework can effectively combine neighboring samples and multiple features for collaborative classification, which could achieve significantly better results than several state-of-the-arts.