Activities such as those involved in food preparation involve interactions between hands, tools and multiple manipulated objects that affect them in visually complex ways making recognition of their constituent actions challenging. We describe a system that classifies action classes in such a setting based on discriminative spatio-temporal superpixel groups. The entire system operates sequentially enabling online action recognition. We obtain state-of-the-art results whilst employing a compact, interpretable representation.