Image Denoising: Invertible and General Denoising Frameworks

Abstract

The widespread use of digital cameras has resulted in a massive number of images being taken every day. However, due to the limitations of sensors and environments such as light conditions, the images are usually contaminated by noise. Obtaining visually clean images are essential for the accuracy of downstream high-level vision tasks. Thus, denoising is a crucial preprocessing step. A fundamental challenge in image denoising is to restore recognizable frequencies in edge and fine-scaled texture regions. Traditional methods usually employ hand-crafted priors to enhance the restoration of these high frequency regions, which seem to be omitted in current deep learning models. We explored whether the clean gradients can be utilized in deep networks as a prior as well as how to incorporate this prior in the networks to boost recovery of missing or obscured picture elements. We present results showing that fusing the pre-denoised images' gradient in the shallow layer contributes to recovering better edges and textures. We also propose a regularization loss term to ensure that the reconstructed images' gradients are close to the clean gradients. Both techniques are indispensable for enhancing the restored image frequencies. We also studied how to make the network preserve input information for better restoration of the high-frequency details. According to the definition of mutual information, we presented that invertibility is indispensable for information losslessness. Then, we proposed the Invertible Restoring Autoencoder (IRAE) network, a multiscale invertible encoder-decoder network. The superiority of this network was verified on three different low-level tasks, image denoising, JPEG image decompression and image inpainting. IRAE showed a good direction to explore more invertible architectures for image restoration. We attempted to further reduce the model size of invertible restoration networks. Our intuition was to use the same learned parameters to encode the noisy images in the forward pass and reconstruct the clean images in the backward pass. However, existing invertible networks use the same distribution for both the input and output obtained in the reversed pass. For our noise removal purpose, the input is noisy, but the reversed output is clean, following two different distributions. It was challenging to design lightweight invertible architectures for denoising. We presented InvDN, converting the noisy input to a clean low-resolution image and a noisy latent representation. To address the challenge mentioned above, we replaced the noisy representation with a clean one random sampled from Gaussian during the reverse pass. InvDN achieved state-of-the-art on real image denoising with much fewer parameters and less run time than existing state-of-the-art models. In addition, InvDN could also generate new noisy images for data augmentation. We also rethought image denoising from a novel aspect and introduced a more general denoising framework. Our framework utilized invertible networks to learn a noisy image distribution, which could be considered as the joint distribution of clean content and noise. The noisy input was mapped to representations in the latent space. A novel disentanglement strategy was applied to the latent representations to obtain the representations for the clean content, which were passed to the reversed network to get the clean image. Since this concept was a novel attempt, we also explored different data augmentation and training strategies for this framework. The proposed FDN was trained and tested from simple to complex tasks on distribution-clear class-specific synthetic noisy datasets, more general remote sensing datasets, and real noisy datasets and achieved competitive results with fewer parameters and faster speed. This work contributed a novel perspective and potential direction to design low-level task models in the future.