This work presents an automatic method for optical flow inpainting. Given a video, each frame domain is endowed with a Riemannian metric based on the video pixel values. The missing optical flow is recovered by solving the Absolutely Minimizing Lipschitz Extension (AMLE) partial differential equation on the Riemannian manifold. An efficient numerical algorithm is proposed using eikonal operators for nonlinear elliptic partial differential equations on a finite graph. The choice of the metric is discussed and the method is applied to optical flow inpainting and sparse-to-dense optical flow estimation, achieving top-tier performance in terms of End-Point-Error (EPE).