Greedy pursuit is one of the standard approaches for sparse approximation. Since the derivation of the conventional greedy pursuit schemes, including matching pursuit (MP) and orthogonal MP (OMP), is based on the inner product space, they may not work properly in the presence of impulsive noise. In this work, we devise a new definition of correlation in lp-space with p>0, called lp-correlation, and introduce the concept of orthogonality in lp-space. Based on the lp-correlation and lp-orthogonality, which are generalizations of the absolute inner product and orthogonality of inner product space, respectively, we develop three greedy pursuit algorithms, namely, lp-MP, lp-OMP, and weak lp-MP, for robust sparse approximation in the presence of outliers.