Enhancing surveillance video captured in inclement weather

Abstract

This work describes a real-time video processing system for improving the visibility of images captured in inclement weather. The system is entirely automatic and is specifically tailored for use with regular surveillance cameras. Estimating what an image captured in fog would have looked like in good visibility conditions requires knowing the depth information of the scene. Since this information is unavailable, the system estimates the enhancement parameters by formulating a probability measure and using an optimization algorithm to find a set of highly probable parameter values. The thesis investigates current state of the art probability formulations and graph-cut based optimization algorithms. A more accurate probability model, which incorporates the expected geometry of a surveillance camera, is shown to be compatible with the a-expansion algorithm and is used to improve the result of the depth estimation process. A new formulation of a-expansion is presented, which means good solutions to convex functions of label difference can now be found efficiently. More importantly, the multilabel swap algorithm provides a flexible trade-off (in terms of solution quality and efficiency) over the range of current multilabel graph-cut algorithms (with the extremes being multilabel encodings and binary moves). Finally, the system implements a robust background estimation algorithm using the graphics processing unit to remove foreground objects from the image in real-time. Since foreground objects are the remaining dominant source of error in the estimation process, the resulting depth estimations induce few (if any) artifacts into the enhanced video.