We propose a fast algorithm for an adaptive variant of the classical bilateral filter, where the range kernel is allowed to vary from pixel to pixel. Several fast and accurate algorithms have been proposed for bilateral filtering, but they assume that the same range kernel is used at each pixel and hence cannot be used for adaptive bilateral filtering (ABF). Only recently, it was shown that fast algorithms for ABF can be developed by approximating the local histogram around each pixel using polynomials. The present algorithm is derived using an entirely different approximation, namely, the range kernels across all pixels are jointly approximated (compressed) using singular value decomposition (SVD). The SVD involves a very large matrix and cannot be computed exactly; however, we are able to get a sufficiently accurate approximation using the Nystrom method (without populating/storing the entire matrix). We show that this SVD-type decomposition allows us to approximate the adaptive bilateral filter using fast convolutions. To demonstrate the speed and accuracy of the proposed algorithm in relation to existing algorithms, we use it for texture filtering, JPEG deblocking, and detail enhancement.