Compact codebook creation and local feature coding in bag-of-features model for image classification

Abstract

Bag-of-features model is an effective framework for generating image representation and has been established as the state-of-the-art for image categorization. In general, it consists of four key modules: local feature extraction, codebook generation, feature coding and feature pooling. In practice, the realization of these four modules is very flexible. While leaving a large room for performance improvement, this flexibility also raises many open issues in the research of bag-of-features model. This thesis focuses on three specific issues. They are 1) How to create a low-dimensional image representation with a few number of visual words? 2) How to manipulate codebook generation and feature coding to create image representations which lead to better classification performance? 3) What is the underlying classification rule of a bag-of-features model based classification system? To answer the first question, a technique called compact codebook creation is explored. The study of this part leads to two pieces of works: a unified probabilistic framework for supervised compact codebook creation and a scalable unsupervised compact codebook creation for the large scale setting. These works not only lead to further improvement over the existing methods but also largely extend the applicability of the compact codebook creation approach. To answer the second question, this thesis employs both theoretical analysis and case studies. More specifically, two general properties of a good encoding system are proposed and three encoding approaches which satisfy these properties are developed. These approaches include: a localized soft-assignment coding method, a multi-projection-multi-codebook scheme for HEp-2 cell image classification and two encoding methods for second-level local features. These studies provide general guidances and practical approaches for designing good encoding systems. At last, to answer the third question, a classification rule visualization tool for the state-of-the-art bag-of-features model is proposed. This tool provides an intuitive way to understand how classification decision is made and gains some insights into the existing bag-of-feature model.