Observers for Scene Reconstruction Using Light-Field Measurements

Abstract

This thesis investigates the task of visual scene reconstruction from a systems theory perspective. In this framework, the scene can be considered as the unknown state to be estimated, and the output of the system is a light-field. While measurements of a light-field can be obtained through more classical sensors such as monocular cameras, light-field cameras offer several advantages for scene reconstruction because the gradients of light-fields are known to be highly correlated with depth. Proving what conditions are necessary in order for depth estimation to be possible has remained a significant theoretical gap in the literature. In this thesis it is shown that for any mildly complex scene class, if depth can be extracted from light-field gradients for any scene in that class, then it is necessary and sufficient that each scene in the class is Lambertian and textured. The geometry of light-field cameras is explored in detail, resulting in a novel bijective point-projection model with clear applications to scene reconstruction that is later used for state-of-the-art camera calibration. The performance of scene reconstruction tasks depends crucially on the way in which the scene is represented. Observers for explicit and implicit scene representations are derived. In both cases, convergence is guaranteed and demonstrated experimentally, but in the latter case, finite-time convergence is derived and under milder conditions, even if the underlying state is infinite-dimensional.