Residual Multiscale Based Single Image Deraining

Abstract

Rain streaks deteriorate the performance of many computer vision algorithms. Previous methods represent rain streaks by different rain layers and then separate those layers from the background image. However, it is rather difficult to decouple a rain image into rain and background layers due to the complexity of real-world rain, such as various shapes, directions, and densities of rain streaks. In this paper, we propose a residual multiscale pyramid based single image deraining method to alleviate the difficulty of rain image decomposition. In particular, we remove rain streaks in a coarse-to-fine manner. In this fashion, the heavy rain can be significantly removed in the coarse-resolution level of the pyramid first, and the light rain will then be further removed in the high-resolution level.This allows us to avoid distinguishing the densities of rain streaks explicitly since the inaccurate classification of rain densities may lead to over- or insufficient-removal of rain. Furthermore, the residual between a recovered image and its corresponding rain image can provide vital clues of rain streaks. We therefore exploit such residual as an attention map for deraining in its consecutive finer-level. Benefiting from the residual attention maps, rain layers can be better extracted from a higher-resolution input image. Extensive experimental results on synthetic and real datasets demonstrate that our method outperforms the state of the art significantly.