Real-time identification of facial expressions is an important topic in the area of human computer interaction and pattern recognition. The research has gained significant attention in recent years. However, many challenges still exist. This is because an individual might display different expressions at different times even for the same mood. Expressions can also be influenced by health. Our proposed framework aims to capture unique information related to facial expressions from salient patches. We extract representative feature patterns at both micro and macro levels, and use a support vector machine (SVM) classifier to label expressions. Our experimental results using the Japanese facial expression (JAFEE) and Cohn-Kanade (CK) datasets achieve high recognition rate and fast computation time, outperforming existing work.