Image based automatic vehicle damage detection

Abstract

Automatically detecting vehicle damage using photographs taken at the accident scene is very useful as it can greatly reduce the cost of processing insurance claims, as well as provide greater convenience for vehicle users. An ideal scenario would be where the vehicle user can upload a few photographs of the damaged car taken from a mobile phone and have the dam- age assessment and insurance claim processing done automatically. However, such a solution remains a challenging task due to a number of factors. For a start, the scene of the accident is typically an unknown and uncontrolled outdoor environment with a plethora of factors beyond our control including scene illumination and the presence of surrounding objects which are not known a priori. In addition, since vehicles have very reflective metallic bodies the photographs taken in such an uncontrolled environment can be expected to have a considerable amount of inter object reflection. Therefore, the application of standard computer vision techniques in this context is a very challenging task. Moreover, solving this task opens up a fascinating repertoire of computer vision problems which need to be addressed in the context of a very challenging scenario. This thesis describes research undertaken to address the problem of au- tomatic vehicle damage detection using photographs. A pipeline addressing a vertical slice of the broad problem is considered while focusing on mild vehicle damage detection. We propose to use 3D CAD models of undamaged vehicles which are used to obtain ground truth information in order to infer what the vehicle with mild damage in the photograph should have looked like, if it had not been damaged. To this end, we develop 3D pose estimation algorithms to register an undamaged 3D CAD model over a photograph of the known dam- aged vehicle. We present a 3D pose estimation method using image gradient information of the photograph and the 3D model projection. We show how the 3D model projection at the recovered 3D pose can be used to identify components of a vehicle in the photograph which may have mild damage. In addition, we present a more robust 3D pose estimation method by minimizing a novel illumination invariant distance measure, which is based on a Mahalanobis distance between attributes of the 3D model projection and the pixels in the photograph. In principle, image edges which are not present in the 3D CAD model projection can be considered to be vehicle damage. However, since the vehicle body is very reflective, there is a large amount of inter object reflection in the photograph which may be misclassified as damage. In order to detect image edges caused by inter object reflection, we propose to apply multi-view geometry techniques on two photographs of the vehicle taken from different view points. To this end, we also develop a robust method to obtain reliable point correspondences across the photographs which are dominated by large reflective and mostly homogeneous regions. The performance of the proposed methods are experimentally evaluated on real photographs using 3D CAD models of varying accuracy. We expect that the research presented in this thesis will provide the groundwork for designing an automatic photograph based vehicle damage de- tection system. Moreover, we hope that our method will provide the foundation for interesting future research. Show more