Intra-category sketch-based image retrieval by matching deformable part models

Yi Li, Tim Hospedales, Yi-Zhe Song and Shaogang Gong

In Proceedings British Machine Vision Conference 2014


An important characteristic of sketches, compared with text, rests with their ability of intrinsically capturing structure and appearance detail of objects. Nonetheless, akin to traditional text-based image retrieval, conventional sketch-based image retrieval (SBIR) principally focuses on retrieving photos of the same category, neglecting the fine-grained characteristics of sketches. In this paper, we further advocate the expressiveness of sketches and examine their efficacy under a novel intra-category SBIR framework. In particular, we study how sketches can be adopted to permit pose-specific retrieval within object categories. Key challenge to this problem is introducing a mid-level sketch representation that not only captures object pose, but also possess the ability to traverse sketch and photo domains. More specifically, we learn deformable part-based model (DPM) as a mid-level representation to discover and encode the various poses and parts in sketch and image domains independently, after which graph matching is utilized to establish component and part-level correspondences across the two domains. We further propose an SBIR dataset that covers the unique aspects of fine-grained SBIR. Through in-depth experiments, we demonstrate the superior performance of our proposed SBIR framework, and showcase its unique ability in pose-specific retrieval.


Poster Session


Extended Abstract (PDF, 1 page, 1.0M)
Paper (PDF, 12 pages, 2.7M)
Bibtex File


Yi Li, Tim Hospedales, Yi-Zhe Song, and Shaogang Gong. Intra-category sketch-based image retrieval by matching deformable part models. Proceedings of the British Machine Vision Conference. BMVA Press, September 2014.


	title = {Intra-category sketch-based image retrieval by matching deformable part models},
	author = {Li, Yi and Hospedales, Tim and Song, Yi-Zhe and Gong, Shaogang},
	year = {2014},
	booktitle = {Proceedings of the British Machine Vision Conference},
	publisher = {BMVA Press},
	editors = {Valstar, Michel and French, Andrew and Pridmore, Tony}
	doi = { }