Zhang, QiLi, HongdongWang, XueWang, Qing2023-12-080920-5691http://hdl.handle.net/1885/308752This paper is concerned with the problem of multi-view 3D reconstruction with an un-calibrated micro-lens array based light field camera. To acquire 3D Euclidean reconstruction, existing approaches commonly apply the calibration with a checkerboard and motion estimation from static scenes in two steps. Self-calibration is the process of simultaneously estimating intrinsic and extrinsic parameters directly from un-calibrated light fields without the help of a checkerboard. While the self-calibration technique for conventional (pinhole) camera is well understood, how to extend it to light field camera remains a challenging task. This is primarily due to the ultra-small baseline of the light field camera. We propose an effective self-calibration method for a light field camera for automatic metric reconstruction without a laborious pre-calibration process. In contrast to conventional self-calibration, we show how such a self-calibration method can be made numerically stable, by exploiting the regularity and measurement redundancies unique for the light field camera. The proposed method is built upon the derivation of a novel ray-space homography constraint (RSHC) using Plücker parameterization as well as a ray-space infinity homography (RSIH). We also propose a new concept of “rays of the absolute conic (RAC)” defined as a special quadric in 5D projective space P5. A set of new equations are established and solved for self-calibration and 3D metric reconstruction specifically designed for a light field camera. We validate the efficacy of the proposed method on both synthetic and real light fields, and have obtained superior results in both accuracy and robustness.The work was supported by NSFC under Grant 61531014, 61801396, 62031023. We thank the editors and reviewers for valuable suggestions on contents and experiments. We also thank Ying Feng for helpful supports on data collection. Qi Zhang was also supported by Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University under CX201919 and China Scholarship Council (CSC).application/pdfen-AU© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2021Light fieldSelf-calibration3D reconstructionRays of the absolute conic (RAC)202110.1007/s11263-021-01516-12022-09-04