Cousin Network Guided Sketch Recognition via Latent Attribute Warehouse

Abstract

We study the problem of sketch image recognition. This prob- lem is plagued with two major challenges: 1) sketch images are often scarce in contrast to the abundance of natural im- ages, rendering the training task difficult, and 2) the signifi- cant domain gap between sketch image and its natural image counterpart makes the task of bridging the two domains chal- lenging. In order to overcome these challenges, in this paper we propose to transfer the knowledge of a network learned from natural images to a sketch network - a new deep net architecture which we term as cousin network. This network guides a sketch-recognition network to extract more relevant features that are close to those of natural images, via adver- sarial training. Moreover, to enhance the transfer ability of the classification model, a sketch-to-image attribute warehouse is constructed to approximate the transformation between the sketch domain and the real image domain. Extensive experi- ments conducted on the TU-Berlin dataset show that the pro- posed model is able to efficiently distill knowledge from natu- ral images and achieves superior performance than the current state of the art. Show more