In this paper, distributed learning is studied using the approach of coreset. In the context of classification, an algorithm of coreset construction is proposed to reduce the redundancy of data and thus the communication requirement, similarly to source coding in traditional data communications. It is shown that the coreset based boosting has a high convergence rate and small sample complexity. Moreover, it is robust to adversary distribution, thus leading to potential applications in distributed learning systems. Both theoretical and numerical analyses are provided to demonstrate the proposed framework.