Spectrum representation, photometric invariants and shape recovery in imaging spectroscopy

Abstract

This thesis addresses the issues of spectrum representation, photometric invariants and shape recovery in multispectral and hyperspectral imagery. We focus on the development of algorithms to recovery shape and photometric invariants from spectral images captured from a single view and the design of a compact spectrum representation for the purpose of recognition, reconstruction and storage of spectra.

Our first contribution is a novel representation of reflectance spectra for imaging spectroscopy, based on the control points resulting from the interpolation of B-Spline curves to multispectral and hyperspectral reflectance data. Since this interpolation is based upon a knot removal scheme in the parameter domain, the representation can exploit the local support of Splines so as to recover a compact representation robust to noise and local perturbation of reflectance data. In addition, this representation permits the manipulation of image data by the use of numerically stable algorithms and methods commonly used to model smooth curves in graphics. The representation presented here also allows pattern recognition and computer vision tasks to be effected on spectra of dissimilar lengths. Furthermore, our Spline-based representation can be applied not only to single spectral reflectance curves but also multispectral and hyperspectral images by providing a common basis for spatially varying spectral reflectance over various materials.

Secondly, we address the problem of photometric invariance in multispectral imaging making use of an optimisation approach based upon the dichromatic model. In this manner, we cast the recovery of the illuminant spectra and the surface reflectance spectra, the shading and the specular factors in a structural optimisation setting. To facilitate the recovery, we make use of the spectral information provided by multispectral imaging and the structure of image patches to formulate an objective function combining the dichromatic error and the smoothness priors for the surfaces under study. The objective function is quite general, allowing the enforcement of alternative smoothness constraints; and the optimisation framework can be extended in a straightforward manner to trichromatic settings. Moreover, the objective function is convex with respect to the subset of variables to be optimised in each alternating step of the minimisation strategy. This gives rise to an optimal closed-form solution for each of the iterations in our algorithm.

Finally, we address the simultaneous recovery of the shape and refractive index of an object from a spectro-polarimetric image captured from a single view. Here, we focus on the diffuse polarisation process occuring at dielectric surfaces due to sub-surface scattering and transmission from the object surface into the air. The diffuse polarisation of the reflection process is modelled by the Fresnel transmission theory. We present a method for estimating the azimuth angle of surface normals from the spectral variation of the phase of polarisation. Moreover, we estimate the zenith angle of surface normals and refractive index simultaneously in a well-posed optimisation framework. We achieve well-posedness by introducing additional constraints to the problem, including the surface integrability and the material dispersion equation. This yields closed-form solutions to both the zenith angle and the refractive index in each iteration. Show more