Learning with Limited Data and Supervision

Abstract

Deep neural networks have been the main driving force of recent successes in machine learning leading to the deployment of these models in a wide range of industries such as healthcare, autonomous driving, and fintech. Despite the great success, these models are known as data-hungry models requiring many labelled training examples and costly computational resources to solve a pre-determined task. Several obstacles limit the applicability of deep learning models in real-world scenarios. First, annotating large-scale training data in tasks such as object localization or segmentation is cumbersome and demands huge time and labor. Second, in real-world scenarios and applications such as field robotics, the models may be required to learn new classes in an ever-changing environment. However, accessing abundant fully labelled training data for novel classes may be infeasible. Therefore, a model needs to adapt to learn novel classes given only a few examples with simple (weak) annotations. Finally, it is known that most modern deep convolutional networks do not have calibrated confidence scores, meaning that the confidence scores they assign to the outcomes do not match the true frequency of those events. These models are of utmost importance to output calibrated prediction scores that the downstream applications can rely upon, especially in safety-critical applications. This thesis focuses on tackling these limitations in deep learning models with applications in Computer Vision. We investigate the task of finding common objects in small image collections and propose an efficient graphical model inference algorithm that utilizes the structure of the problem to reduce the computational time compared to traditional inference algorithms significantly. We also propose a probabilistic approach to solve the few-shot common object localization problem based on a parametric distribution of each class on a unit sphere. We further extend our model to localize objects of novel classes in unseen images. In the next step, we study pairwise similarity knowledge transfer for weakly supervised object localization to reduce the cost of labor and time in annotating large-scale object detection datasets for novel classes. We learn the similarity functions and the assignment of proposals to different novel classes jointly using alternating optimization and show that the assignment problem becomes an integer linear program for a certain type of loss function. Furthermore, we propose an efficient inference algorithm to overcome the difficulty of computing all pairwise similarities. Finally, to overcome pre-trained models' accuracy degradation in learning expressive probability calibration functions using small calibration data, we introduce and formalize the notion of order-preserving functions. We also present two sub-families of order-preserving functions that benefit from parameter sharing across different classes in classification problems. Show more