Geometry Aware Deep Metric Learning

Abstract

A diverse range of applications in computer vision benefit from the data representations which are dense and compact, yet discriminative enough to learn the subtle changes in the data. Such representation learning seems necessary especially in the Zero Shot Learning applications where the train and the test classes are mutually exclusive. In other words, the learned representations should be discriminative enough to identify the minute cues in the data samples such that the unseen data can be properly categorized accordingly. With the advent of Deep Neural Networks over the last few years, several metric learning algorithms have been developed to address the aforementioned challenging objective. These algorithms learn the embedding space whilst considering the relative similarity/dissimilarity relationships between the data points across the various classes. Although successful, they suffer from a number of serious drawbacks, some of which have been addressed in this thesis.

As our first objective, we extended two popular optimizers, namely Stochastic Gradient Descent with Momentum (SGD-M) and RMSProp, to their respective Riemannian counterparts. Such extension deems necessary while trying to optimize a model under the constrained problem settings. Our proposal reaps the benefits of standard manifold operations while optimizing the parameters of the network that are constrained to reside on a Riemannian manifold. The experimental evaluations vividly showed that the constrained optimizers clearly outperform their non-constrained equivalents over a wide range of datasets and application settings with regards to the improved learning of the embedding space.

We then turn our attention to the general training protocol of Siamese Neural Networks (SiNNs), and address a major yet obvious drawback in its training practice. SiNNs are characterized by a Positive Semi Definite (PSD) matrix M which is invariant to the action of the orthogonal group O(p); thereby resulting in an equivalence class of solutions for M. Taking such invariances into account, we proposed a novel matrix manifold qConv and used it along with the popular Stiefel manifold to exploit the invariances in the siamese networks. We made use of our constrained optimizers to optimize over these two manifolds. Our empirical evaluations clearly showed that the training of SiNNs benefit by invoking such geometrical constraints over the search space whilst making use of such invariances inherent in SiNNs. As our final contribution, we designed and developed a novel, yet effective loss function that incorporates class-wise dissimilarity relationships in learning a discriminative embedding space. Such class-wise dissimilarity relationships have not been considered in the loss functions developed till now; thereby resulting in learning of a sub-optimal embedding space. Hence, we have integrated and maximized such dissimilarity constraints using two standard variants of Sinkhorn Divergences. Further, our experimental evaluations signified the importance of enforcing such constraints in learning a superior embedding space in the presence and absence of noise.