A General Dense Image Matching Framework Combining Direct and Feature-Based Costs

Jim Braux-Zin (1) , Romain Dupont (1) , Adrien Bartoli (2, 3)

1 LVIC - Laboratoire Vision et Ingénierie des Contenus
2 ISIT - Image Science for Interventional Techniques
3 IP - Institut Pascal

Dense motion field estimation (typically optical flow, stereo disparity and surface registration) is a key computer vision problem. Many solutions have been proposed to compute small or large displacements, narrow or wide baseline stereo disparity, but a unified methodology is still lacking. We here introduce a general framework that robustly combines direct and feature-based matching. The feature-based cost is built around a novel robust distance function that handles key points and ``weak'' features such as segments. It allows us to use putative feature matches which may contain mismatches to guide dense motion estimation out of local minima. Our framework uses a robust direct data term (AD-Census). It is implemented with a powerful second order Total Generalized Variation regularization with external and self-occlusion reasoning. Our framework achieves state of the art performance in several cases (standard optical flow benchmarks, wide-baseline stereo and non-rigid surface registration). Our framework has a modular design that customizes to specific application needs.

Domains: Signal and Image processing

Jim Braux-Zin, Romain Dupont, Adrien Bartoli. A General Dense Image Matching Framework Combining Direct and Feature-Based Costs. IEEE International Conference on Computer Vision (ICCV), Dec 2013, Sydney, Australia. ⟨10.1109/ICCV.2013.30⟩. ⟨cea-01836523⟩