Assessing Model Robustness in Complex Visual Environments

Abstract

As machine learning and computer vision technologies evolve and operate in broader applications, assessing model robustness across various complex environments has become increasingly crucial. Unlike traditional evaluation schemes calculating model accuracy on the in-distribution test set, the focus of model robustness assessment shifts towards evaluating the resilience and adaptability of models under various challenging conditions, such as with visual factor variations and even malicious attacks. Due to the wide range and complexity of environments, many aspects in this field remain unexplored or require further development. This thesis aims to address existing gaps by focusing on improving the assessment of model robustness in complex visual environments from multiple perspectives.

The initial part of this thesis focuses on the creation of diverse benchmarks for assessing model robustness, specifically targeting model adaptation and generalization, respectively. This focus arises due to either the absence of suitable benchmarks or the insufficiency of existing benchmarks in capturing the wide range of scenarios. In Chapters 2 and 3, we introduce Alice benchmarks and CIFAR-10-Warehouse for evaluating 1) the adaptation ability of models trained on synthetic data to real-world conditions, and 2) the generalization of models in various environments, respectively. The Alice benchmarks utilize synthetic and real-world data to comprehensively explore Syn2Real domain adaptation challenges. Meanwhile, CIFAR-10-Warehouse, including 180 diverse domains, focuses on broadening the evaluation and enhancing understanding of domain generalization and model accuracy prediction across various out-of-distribution environments.

Then, this thesis explores model robustness in two new scenarios: resilience to visual factor variations and generalizing ability in new unlabeled environments. In Chapter 4, we focus on quantitatively assessing how changes in visual factors impact model robustness, with a particular emphasis on viewpoint. We conduct an in-depth analysis of the influence of pedestrian viewpoint variations on re-identification systems by using the PersonX synthetic data engine for generating controlled data. Chapter 5 studies a new problem of ranking models in unlabeled new environments, which is important for model selection and deployment. It explores different strategies, including using the relationship between model in-distribution performance and out-of-distribution robustness. Furthermore, we propose a "target proxy" method for predicting model performance in unseen environments. These two chapters contribute new perspectives and methods to understanding and improving model robustness evaluation in diverse environments.

Finally, this thesis examines model vulnerability to information leakage attacks, a critical component of model robustness. Specifically, Chapter 6 focuses on the risk of information leakage under reconstruction attacks. In this chapter, we comprehensively evaluate how well traditional metrics reflect human perceptions regarding the leakage of private information in reconstructed images and introduce a novel metric for closer alignment with human judgment.

In conclusion, this thesis not only riches existing benchmark gaps in model robustness evaluation but also explores new challenges in model resilience assessment. By comprehensively testing models against domain shifts, visual factor variations and security threats, this thesis lays a solid foundation for future studies aimed at creating more resilient and trustworthy computer vision systems. Show more