Zdenek Kalal, Krystian Mikolajczyk, and Jirí Matas. Forward-backward error: Automatic detection of tracking failures. In Roy Sterritt, editor, ICPR'2010: Proceedings of the 20th International Conference on Pattern Recognition, pages 2756-2760, 10662 Los Vaqueros Circle, Los Alamitos, USA, August 2010. IEEE Computer Society.
This paper proposes a novel method for tracking failure detection. The detection is based on the Forward-Backward error, i.e. the tracking is performed forward and backward in time and the discrepancies between these two trajectories are measured. We demonstrate that the proposed error enables reliable detection of tracking failures and selection of reliable trajectories in video sequences. We demonstrate that the approach is complementary to commonly used normalized cross-correlation (NCC). Based on the error, we propose a novel object tracker called Median Flow. State-of-the-art performance is achieved on challenging benchmark video sequences which include non-rigid objects.