Site Header Image

Block-based motion estimation for missing video frame interpolation, and spatially scalable (multi-resolution) video coding

Posted on by 1qs9y

Video frames are often dropped during compression at very low bit rates. At the decoder, a missing frame interpolation method synthesizes the missed frames. We propose a two step motion estimation method for the interoplation. More specifically, the coarse motion vector field is refined at the decoder using mesh-based motion estimation instead of using computationally intensive dense motion estimation. We propose a framework for detecting and utilizing local motion boundaries in terms of an explicit model. Motion boundaries are modeled using edge detection and Hough transform. The motion of the occluding side is represented by affine mapping. The newly appearing region is also detected. Each pixel is interpolated differently according to adaptive interpolation based on the property of the pixel: moving, static, and disoccluded. The resulting quality of interpolated images is constantly better than block-based interpolation and is comparable to optical flow methods.

Results

We implemented the proposed algorithm using various QCIF (176×144) video sequences. Fig. 1(a) shows a typical motion vector field resulting from the block matching algorithm (BMA). The block size for the BMA is set to 16×16. We tried to make the number of triangles of a uniform mesh the same as the case of the BMA. After fixing the locations of node points, the algorithm obtains one motion vector per node in order to estimate affine parameters per triangle. Fig. 1(b) shows the improved motion vector field. Fig. 2 shows comparisons among the block-based method, the pel-recursive method, and the proposed method. The frame numbers in Fig. 2 refer to the interpolated frames. It means that we do not include the PSNRs of the transmitted frames in the graphs. The proposed algorithm shows a performance close to the pel-recursive approach in terms of PSNR. Fig. 3 shows a comparison between two interpolated images using the block-based method and the proposed method, respectively.