Spectral dichromatic parameter recovery from two views via total variation hyper-priors

Abstract

In this paper, we propose an approach for the recovery of the dichromatic model from two hyperspectral or multispectral images, i.e., the joint estimation of illuminant, reflectance, and shading of each pixel, as well as the optical flow between the two views. The approach is based on the minimization of an energy functional linking the dichromatic model to the image appearances and the flow between the images to the factorized reflectance component. In order to minimize the resulting under-constrained problem, we apply vectorial total variation regularizers both to the scene reflectance, and to the flow hyper-parameters. We do this by enforcing the physical priors for the reflectance of the materials in the scene and assuming the flow varies smoothly within rigid objects in the image. We show the effectiveness of the approach compared with single view model recovery both in terms of model constancy and of closeness to the ground truth