Motion Detail Preserving Optical Flow Estimation

    13 Votes

Optical flow is the apparent motion of brightness patterns in the image. Ideally, optical flow would be the same as the motion field. The motion field is the projection into the image of three dimensional motion vectors. In conventional optical flow, Dominant Scheme - Coarse to Fine Warping Framework is used. The input image is represented as a tree of regions.

The optical flow is estimated by optimizing an energy function. Optical flow estimation on the coarser level region tree is used for defining region wise finer displacement samplings for finer level region trees. It uses Middlebury optical flow evaluation.

Multi Scale Problem In Coarse-to-fine Warping

Multi Scale Problem In Coarse-to-fine Warping
In image A and B there are two input patches. In figure C, Flow estimate using the coarse-to-fine variational setting. In image D, Our flow estimate. In figure E and F, Two consecutive levels in the pyramid. Flow fields are visualized using the colour code. Large displacement optical  flow may not be well estimated. Inclination to diminish small motion structures when spatially significant and abrupt change of the displacement exists. Solution to this is by improving flow initialization to reduce the reliance of the initialization from coarser levels and enables recovering many motion details at each scale.


Extended coarse-to-fine motion estimation for both large and small displacement optical flow


A new data term to selectively combine constraints


Efficient numerical solver for discrete continuous optimization

Optical Flow Model - Robust Data Function

Objective function for  development of a new optimization procedure. U denotes the flow field that represents the displacement between frames  I1 and I2 ,x represents the 2D coordinates.

Data Constraints

Colour constraint, D1(u,x)= I2(x+u) - I1(x)

Extended Flow Initialization

Finding multiple extended displacements (denoted as {u0,u1,….,un}) to improve estimation in Uc. Uc which is the flow field computed in the immediately coarser level. The steps adopted to obtain the extended displacements.

  • SIFT Feature Detection
  • Selection
  • Expansion
  • Patch Matching   
  • Matching Field Fusion

SIFT Feature Detection

SIFT feature detection and matching can efficiently capture large motion for objects undergoing translational and rotational motion. Employ only the  sparse matching of discriminative points, which avoids introducing many ambiguous correspondences and outliers. Employ discrete optimization to only select the most credible candidates. During Patch Matching, sometimes it still misses some motion vectors,because small texture-less objects may not have distinct features.

SIFT descriptors, the patches on which they operate should at least contain 16 x 16 samples as suggested. Compute the matching field Un by minimizing energy. Total of five colour and gradient channels used. Noise can be quickly rejected in the following optimization step with the collection of a set of flow candidates for each pixel.

Download this file (MOTION  DETAIL  PRESERVING OPTICAL  FLOW  ESTIMATION.ppt)Motion Detail Preserving Optical Flow Estimation[PPT Presentation]1613 Kb