Video Segmentation by Non-Local Consensus voting

Alon Faktor and Michal Irani

In Proceedings British Machine Vision Conference 2014


We address the problem of Foregrond/Background segmentation of ``unconstrained'' video. By ``unconstrained'' we mean that the moving objects and the background scene may be highly non-rigid (e.g., waves in the sea); the camera may undergo a complex motion with 3D parallax; moving objects may suffer from motion blur, large scale and illumination changes, etc. Most existing segmentation methods fail on such unconstrained videos, especially in the presence of highly non-rigid motion and low resolution. We propose a computationally efficient algorithm which is able to produce accurate results on a large variety of unconstrained videos. This is obtained by casting the video segmentation problem as a voting scheme on the graph of co-occurring regions in the video sequence. We start from crude saliency votes at each pixel, and iteratively correct those votes by `consensus voting' of co-occurring regions across the video sequence. The power of our consensus voting comes from the non-locality of the region co-occurrence, both in space and in time -- enabling propagation of diverse and rich information across the entire video sequence. Qualitative and quantitative experiments indicate that our approach outperforms current state-of-the-art methods.


Segmentation and Object Detection


Extended Abstract (PDF, 1 page, 515K)
Paper (PDF, 12 pages, 1.5M)
Bibtex File



Alon Faktor, and Michal Irani. Video Segmentation by Non-Local Consensus voting. Proceedings of the British Machine Vision Conference. BMVA Press, September 2014.


	title = {Video Segmentation by Non-Local Consensus voting},
	author = {Faktor, Alon and Irani, Michal},
	year = {2014},
	booktitle = {Proceedings of the British Machine Vision Conference},
	publisher = {BMVA Press},
	editors = {Valstar, Michel and French, Andrew and Pridmore, Tony}
	doi = { }